Implement the following plan:

# Plan: Improve Group-By Stats Query Performance

## Context

The stats endpoint (`JobsStatistics` GraphQL query) times out for large job counts. A typical stats page with grouping (by user/project/cluster) triggers **3 separate full-table-scan + GROUP BY queries** against the job table:
1. `JobsStatsGrouped` — main aggregation (totalJobs, walltime, node/core/acc hours)
2. `AddJobCountGrouped("short")` — second full scan for short-running jobs count
3. `AddJobCountGrouped("running")` — third full scan for running jobs count

Each query independently scans and aggregates every row matching the filter. With millions of jobs, this is prohibitively slow.

Additional minor inefficiencies:
- Unnecessary `LEFT JOIN hpc_user` even when grouping by project/cluster/subcluster
- O(n × bins) loop matching in duration histogram bin assignment (should be O(1) map lookup)
- `time.Now().Unix()` called multiple times in a single query build

## Critical Files

- `internal/repository/stats.go` — all stats/histogram logic
- `internal/graph/schema.resolvers.go` — GraphQL resolver that orchestrates query calls (lines 655–711)

## Implementation

### Fix 1: Merge 3 queries into 1 (biggest impact)

Modify `buildStatsQuery` to add `runningJobs` and `shortJobs` as inline conditional counts:

```sql
COUNT(CASE WHEN job.job_state = 'running' THEN 1 END) as runningJobs,
COUNT(CASE WHEN job.duration < :shortThreshold THEN 1 END) as shortJobs
```

- `buildStatsQuery(filter, col)` gains a `shortThreshold int` parameter and always returns these two extra columns
- `JobsStatsGrouped` and `JobsStats` scan the new columns and populate `RunningJobs`/`ShortJobs` on each result
- In `schema.resolvers.go`, remove the separate `AddJobCountGrouped` / `AddJobCount` calls when the main stats were already fetched (i.e., when `requireField(ctx, "totalJobs")` was true)
- The separate `buildCountQuery` / `JobCountGrouped` / `AddJobCountGrouped` / `AddJobCount` paths are kept for the edge case where ONLY `runningJobs`/`shortJobs` are requested without main stats fields

### Fix 2: Remove unnecessary LEFT JOIN for non-user grouping

In `buildStatsQuery`, only add `.LeftJoin("hpc_user ON hpc_user.username = job.hpc_user")` and select `name` when `col == "job.hpc_user"`. For project/cluster/subcluster grouping, omit the join and select `'' as name` or remove that column entirely.

This eliminates a join that always returns NULL for 3 of the 4 grouping dimensions.

### Fix 3: Fix O(n × bins) histogram bin matching

In `jobsDurationStatisticsHistogram` (stats.go:848–853), replace the inner loop over `points` with a map lookup:

```go
binMap := make(map[int]int, len(points))
for i, p := range points {
    binMap[p.Value] = i
}
// then in rows.Next():
if idx, ok := binMap[point.Value * binSizeSeconds]; ok {
    points[idx].Count = point.Count
}
```

Same fix applies to `jobsMetricStatisticsHistogram` (stats.go:988–993).

### Fix 4: Compute timestamp once per query build

In `buildStatsQuery`, call `time.Now().Unix()` once at the top of the function and reuse the variable in all 4–5 CASE WHEN expressions. Minor correctness and readability improvement.

## Resolver Change Summary

`schema.resolvers.go` around line 667–681:

```go
// Before: 3 separate queries
stats, err = r.Repo.JobsStatsGrouped(...)   // query 1
stats, err = r.Repo.AddJobCountGrouped(..., "short")   // query 2
stats, err = r.Repo.AddJobCountGrouped(..., "running") // query 3

// After: 1 query (runningJobs/shortJobs already in stats result)
stats, err = r.Repo.JobsStatsGrouped(...)   // query 1 only
// AddJobCountGrouped calls removed when main stats were fetched
```

## Verification

1. Build: `go build ./...`
2. Run existing tests: `go test ./internal/repository/...`
3. Manual test: start server with `./cc-backend -server -dev`, open the analysis/stats page with a large cluster and verify it loads without timeout
4. Check debug log output for `Timer JobsStatsGrouped` — should drop significantly (1 query vs 3)

If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /Users/jan/.claude/projects/-Users-jan-prg-ClusterCockpit-cc-backend/be115332-505c-4b8f-acdf-f8de6ddbed14.jsonl


ToolSearch

Question: select:Read,Grep,Edit


Question: select:Read,Grep,Edit


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→// This file contains job statistics and histogram generation functionality for the JobRepository.
     7→//
     8→// # Job Statistics
     9→//
    10→// The statistics methods provide aggregated metrics about jobs including total jobs, users,
    11→// walltime, and resource usage (nodes, cores, accelerators). Statistics can be computed:
    12→//   - Overall (JobsStats): Single aggregate across all matching jobs
    13→//   - Grouped (JobsStatsGrouped): Aggregated by user, project, cluster, or subcluster
    14→//   - Counts (JobCountGrouped, AddJobCount): Simple job counts with optional filtering
    15→//
    16→// All statistics methods support filtering via JobFilter and respect security contexts.
    17→//
    18→// # Histograms
    19→//
    20→// Histogram methods generate distribution data for visualization:
    21→//   - Duration, nodes, cores, accelerators (AddHistograms)
    22→//   - Job metrics like CPU load, memory usage (AddMetricHistograms)
    23→//
    24→// Histograms use intelligent binning:
    25→//   - Duration: Variable bin sizes (1m, 10m, 1h, 6h, 12h, 24h) with zero-padding
    26→//   - Resources: Natural value-based bins
    27→//   - Metrics: Normalized to peak values with configurable bin counts
    28→//
    29→// # Running vs. Completed Jobs
    30→//
    31→// Statistics handle running jobs specially:
    32→//   - Duration calculated as (now - start_time) for running jobs
    33→//   - Metric histograms for running jobs load data from metric backend instead of footprint
    34→//   - Job state filtering distinguishes running/completed jobs
    35→//
    36→// # Performance Considerations
    37→//
    38→// - All queries use prepared statements via stmtCache
    39→// - Complex aggregations use SQL for efficiency
    40→// - Histogram pre-initialization ensures consistent bin ranges
    41→// - Metric histogram queries limited to 5000 jobs for running job analysis
    42→
    43→package repository
    44→
    45→import (
    46→	"context"
    47→	"database/sql"
    48→	"fmt"
    49→	"time"
    50→
    51→	"github.com/ClusterCockpit/cc-backend/internal/config"
    52→	"github.com/ClusterCockpit/cc-backend/internal/graph/model"
    53→	"github.com/ClusterCockpit/cc-backend/internal/metricdispatch"
    54→	"github.com/ClusterCockpit/cc-backend/pkg/archive"
    55→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    56→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    57→	sq "github.com/Masterminds/squirrel"
    58→)
    59→
    60→// groupBy2column maps GraphQL Aggregate enum values to their corresponding database column names.
    61→// Used by JobsStatsGrouped and JobCountGrouped to translate user-facing grouping dimensions
    62→// into SQL GROUP BY clauses. GraphQL validation ensures only valid enum values are accepted.
    63→var groupBy2column = map[model.Aggregate]string{
    64→	model.AggregateUser:       "job.hpc_user",
    65→	model.AggregateProject:    "job.project",
    66→	model.AggregateCluster:    "job.cluster",
    67→	model.AggregateSubcluster: "job.subcluster",
    68→}
    69→
    70→// sortBy2column maps GraphQL SortByAggregate enum values to their corresponding computed column names.
    71→// Used by JobsStatsGrouped to translate sort preferences into SQL ORDER BY clauses.
    72→// Column names match the AS aliases used in buildStatsQuery.
    73→var sortBy2column = map[model.SortByAggregate]string{
    74→	model.SortByAggregateTotaljobs:      "totalJobs",
    75→	model.SortByAggregateTotalusers:     "totalUsers",
    76→	model.SortByAggregateTotalwalltime:  "totalWalltime",
    77→	model.SortByAggregateTotalnodes:     "totalNodes",
    78→	model.SortByAggregateTotalnodehours: "totalNodeHours",
    79→	model.SortByAggregateTotalcores:     "totalCores",
    80→	model.SortByAggregateTotalcorehours: "totalCoreHours",
    81→	model.SortByAggregateTotalaccs:      "totalAccs",
    82→	model.SortByAggregateTotalacchours:  "totalAccHours",
    83→}
    84→
    85→// buildCountQuery constructs a SQL query to count jobs with optional grouping and filtering.
    86→//
    87→// Parameters:
    88→//   - filter: Job filters to apply (cluster, user, time range, etc.)
    89→//   - kind: Special filter - "running" for running jobs only, "short" for jobs under threshold
    90→//   - col: Column name to GROUP BY; empty string for total count without grouping
    91→//
    92→// Returns a SelectBuilder that produces either:
    93→//   - Single count: COUNT(job.id) when col is empty
    94→//   - Grouped counts: col, COUNT(job.id) when col is specified
    95→//
    96→// The kind parameter enables counting specific job categories:
    97→//   - "running": Only jobs with job_state = 'running'
    98→//   - "short": Only jobs with duration < ShortRunningJobsDuration config value
    99→//   - empty: All jobs matching filters
   100→func (r *JobRepository) buildCountQuery(
   101→	filter []*model.JobFilter,
   102→	kind string,
   103→	col string,
   104→) sq.SelectBuilder {
   105→	var query sq.SelectBuilder
   106→
   107→	if col != "" {
   108→		query = sq.Select(col, "COUNT(job.id)").From("job").GroupBy(col)
   109→	} else {
   110→		query = sq.Select("COUNT(job.id)").From("job")
   111→	}
   112→
   113→	switch kind {
   114→	case "running":
   115→		query = query.Where("job.job_state = ?", "running")
   116→	case "short":
   117→		query = query.Where("job.duration < ?", config.Keys.ShortRunningJobsDuration)
   118→	}
   119→
   120→	for _, f := range filter {
   121→		query = BuildWhereClause(f, query)
   122→	}
   123→
   124→	return query
   125→}
   126→
   127→// buildStatsQuery constructs a SQL query to compute comprehensive job statistics with optional grouping.
   128→//
   129→// Parameters:
   130→//   - filter: Job filters to apply (cluster, user, time range, etc.)
   131→//   - col: Column name to GROUP BY; empty string for overall statistics without grouping
   132→//
   133→// Returns a SelectBuilder that produces comprehensive statistics:
   134→//   - totalJobs: Count of jobs
   135→//   - totalUsers: Count of distinct users (always 0 when grouping by user)
   136→//   - totalWalltime: Sum of job durations in hours
   137→//   - totalNodes: Sum of nodes used across all jobs
   138→//   - totalNodeHours: Sum of (duration × num_nodes) in hours
   139→//   - totalCores: Sum of hardware threads used across all jobs
   140→//   - totalCoreHours: Sum of (duration × num_hwthreads) in hours
   141→//   - totalAccs: Sum of accelerators used across all jobs
   142→//   - totalAccHours: Sum of (duration × num_acc) in hours
   143→//
   144→// Special handling:
   145→//   - Running jobs: Duration calculated as (now - start_time) instead of stored duration
   146→//   - Grouped queries: Also select grouping column and user's display name from hpc_user table
   147→//   - All time values converted from seconds to hours (÷ 3600) and rounded
   148→func (r *JobRepository) buildStatsQuery(
   149→	filter []*model.JobFilter,
   150→	col string,
   151→) sq.SelectBuilder {
   152→	var query sq.SelectBuilder
   153→
   154→	if col != "" {
   155→		query = sq.Select(
   156→			col,
   157→			"name",
   158→			"COUNT(job.id) as totalJobs",
   159→			"COUNT(DISTINCT job.hpc_user) AS totalUsers",
   160→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END)) / 3600) as int) as totalWalltime`, time.Now().Unix()),
   161→			`CAST(SUM(job.num_nodes) as int) as totalNodes`,
   162→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_nodes) / 3600) as int) as totalNodeHours`, time.Now().Unix()),
   163→			`CAST(SUM(job.num_hwthreads) as int) as totalCores`,
   164→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_hwthreads) / 3600) as int) as totalCoreHours`, time.Now().Unix()),
   165→			`CAST(SUM(job.num_acc) as int) as totalAccs`,
   166→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_acc) / 3600) as int) as totalAccHours`, time.Now().Unix()),
   167→		).From("job").LeftJoin("hpc_user ON hpc_user.username = job.hpc_user").GroupBy(col)
   168→	} else {
   169→		query = sq.Select(
   170→			"COUNT(job.id) as totalJobs",
   171→			"COUNT(DISTINCT job.hpc_user) AS totalUsers",
   172→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END)) / 3600) as int)`, time.Now().Unix()),
   173→			`CAST(SUM(job.num_nodes) as int)`,
   174→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_nodes) / 3600) as int)`, time.Now().Unix()),
   175→			`CAST(SUM(job.num_hwthreads) as int)`,
   176→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_hwthreads) / 3600) as int)`, time.Now().Unix()),
   177→			`CAST(SUM(job.num_acc) as int)`,
   178→			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_acc) / 3600) as int)`, time.Now().Unix()),
   179→		).From("job")
   180→	}
   181→
   182→	for _, f := range filter {
   183→		query = BuildWhereClause(f, query)
   184→	}
   185→
   186→	return query
   187→}
   188→
   189→// JobsStatsGrouped computes comprehensive job statistics grouped by a dimension (user, project, cluster, or subcluster).
   190→//
   191→// This is the primary method for generating aggregated statistics views in the UI, providing
   192→// metrics like total jobs, walltime, and resource usage broken down by the specified grouping.
   193→//
   194→// Parameters:
   195→//   - ctx: Context for security checks and cancellation
   196→//   - filter: Filters to apply (time range, cluster, job state, etc.)
   197→//   - page: Optional pagination (ItemsPerPage: -1 disables pagination)
   198→//   - sortBy: Optional sort column (totalJobs, totalWalltime, totalCoreHours, etc.)
   199→//   - groupBy: Required grouping dimension (User, Project, Cluster, or SubCluster)
   200→//
   201→// Returns a slice of JobsStatistics, one per group, with:
   202→//   - ID: The group identifier (username, project name, cluster name, etc.)
   203→//   - Name: Display name (for users, from hpc_user.name; empty for other groups)
   204→//   - Statistics: totalJobs, totalUsers, totalWalltime, resource usage metrics
   205→//
   206→// Security: Respects user roles via SecurityCheck - users see only their own data unless admin/support.
   207→// Performance: Results are sorted in SQL and pagination applied before scanning rows.
   208→func (r *JobRepository) JobsStatsGrouped(
   209→	ctx context.Context,
   210→	filter []*model.JobFilter,
   211→	page *model.PageRequest,
   212→	sortBy *model.SortByAggregate,
   213→	groupBy *model.Aggregate,
   214→) ([]*model.JobsStatistics, error) {
   215→	start := time.Now()
   216→	col := groupBy2column[*groupBy]
   217→	query := r.buildStatsQuery(filter, col)
   218→
   219→	query, err := SecurityCheck(ctx, query)
   220→	if err != nil {
   221→		return nil, err
   222→	}
   223→
   224→	if sortBy != nil {
   225→		sortBy := sortBy2column[*sortBy]
   226→		query = query.OrderBy(fmt.Sprintf("%s DESC", sortBy))
   227→	}
   228→	if page != nil && page.ItemsPerPage != -1 {
   229→		limit := uint64(page.ItemsPerPage)
   230→		query = query.Offset((uint64(page.Page) - 1) * limit).Limit(limit)
   231→	}
   232→
   233→	rows, err := query.RunWith(r.DB).QueryContext(ctx)
   234→	if err != nil {
   235→		cclog.Warn("Error while querying DB for job statistics")
   236→		return nil, err
   237→	}
   238→	defer rows.Close()
   239→
   240→	stats := make([]*model.JobsStatistics, 0, 100)
   241→
   242→	for rows.Next() {
   243→		var id sql.NullString
   244→		var name sql.NullString
   245→		var jobs, users, walltime, nodes, nodeHours, cores, coreHours, accs, accHours sql.NullInt64
   246→		if err := rows.Scan(&id, &name, &jobs, &users, &walltime, &nodes, &nodeHours, &cores, &coreHours, &accs, &accHours); err != nil {
   247→			cclog.Warnf("Error while scanning rows: %s", err.Error())
   248→			return nil, err
   249→		}
   250→
   251→		if id.Valid {
   252→			var totalJobs, totalUsers, totalWalltime, totalNodes, totalNodeHours, totalCores, totalCoreHours, totalAccs, totalAccHours int
   253→			var personName string
   254→
   255→			if name.Valid {
   256→				personName = name.String
   257→			}
   258→
   259→			if jobs.Valid {
   260→				totalJobs = int(jobs.Int64)
   261→			}
   262→
   263→			if users.Valid {
   264→				totalUsers = int(users.Int64)
   265→			}
   266→
   267→			if walltime.Valid {
   268→				totalWalltime = int(walltime.Int64)
   269→			}
   270→
   271→			if nodes.Valid {
   272→				totalNodes = int(nodes.Int64)
   273→			}
   274→			if cores.Valid {
   275→				totalCores = int(cores.Int64)
   276→			}
   277→			if accs.Valid {
   278→				totalAccs = int(accs.Int64)
   279→			}
   280→
   281→			if nodeHours.Valid {
   282→				totalNodeHours = int(nodeHours.Int64)
   283→			}
   284→			if coreHours.Valid {
   285→				totalCoreHours = int(coreHours.Int64)
   286→			}
   287→			if accHours.Valid {
   288→				totalAccHours = int(accHours.Int64)
   289→			}
   290→
   291→			if col == "job.hpc_user" {
   292→				// name := r.getUserName(ctx, id.String)
   293→				stats = append(stats,
   294→					&model.JobsStatistics{
   295→						ID:             id.String,
   296→						Name:           personName,
   297→						TotalJobs:      totalJobs,
   298→						TotalWalltime:  totalWalltime,
   299→						TotalNodes:     totalNodes,
   300→						TotalNodeHours: totalNodeHours,
   301→						TotalCores:     totalCores,
   302→						TotalCoreHours: totalCoreHours,
   303→						TotalAccs:      totalAccs,
   304→						TotalAccHours:  totalAccHours,
   305→					})
   306→			} else {
   307→				stats = append(stats,
   308→					&model.JobsStatistics{
   309→						ID:             id.String,
   310→						TotalJobs:      totalJobs,
   311→						TotalUsers:     totalUsers,
   312→						TotalWalltime:  totalWalltime,
   313→						TotalNodes:     totalNodes,
   314→						TotalNodeHours: totalNodeHours,
   315→						TotalCores:     totalCores,
   316→						TotalCoreHours: totalCoreHours,
   317→						TotalAccs:      totalAccs,
   318→						TotalAccHours:  totalAccHours,
   319→					})
   320→			}
   321→		}
   322→	}
   323→
   324→	if err := rows.Err(); err != nil {
   325→		return nil, err
   326→	}
   327→
   328→	cclog.Debugf("Timer JobsStatsGrouped %s", time.Since(start))
   329→	return stats, nil
   330→}
   331→
   332→// JobsStats computes overall job statistics across all matching jobs without grouping.
   333→//
   334→// This method provides a single aggregate view of job metrics, useful for dashboard
   335→// summaries and overall system utilization reports.
   336→//
   337→// Parameters:
   338→//   - ctx: Context for security checks and cancellation
   339→//   - filter: Filters to apply (time range, cluster, job state, etc.)
   340→//
   341→// Returns a single-element slice containing aggregate statistics:
   342→//   - totalJobs, totalUsers, totalWalltime
   343→//   - totalNodeHours, totalCoreHours, totalAccHours
   344→//
   345→// Unlike JobsStatsGrouped, this returns overall totals without breaking down by dimension.
   346→// Security checks are applied via SecurityCheck to respect user access levels.
   347→func (r *JobRepository) JobsStats(
   348→	ctx context.Context,
   349→	filter []*model.JobFilter,
   350→) ([]*model.JobsStatistics, error) {
   351→	start := time.Now()
   352→	query := r.buildStatsQuery(filter, "")
   353→	query, err := SecurityCheck(ctx, query)
   354→	if err != nil {
   355→		return nil, err
   356→	}
   357→
   358→	row := query.RunWith(r.DB).QueryRowContext(ctx)
   359→	stats := make([]*model.JobsStatistics, 0, 1)
   360→
   361→	var jobs, users, walltime, nodes, nodeHours, cores, coreHours, accs, accHours sql.NullInt64
   362→	if err := row.Scan(&jobs, &users, &walltime, &nodes, &nodeHours, &cores, &coreHours, &accs, &accHours); err != nil {
   363→		cclog.Warn("Error while scanning rows")
   364→		return nil, err
   365→	}
   366→
   367→	if jobs.Valid {
   368→		var totalNodeHours, totalCoreHours, totalAccHours int
   369→
   370→		if nodeHours.Valid {
   371→			totalNodeHours = int(nodeHours.Int64)
   372→		}
   373→		if coreHours.Valid {
   374→			totalCoreHours = int(coreHours.Int64)
   375→		}
   376→		if accHours.Valid {
   377→			totalAccHours = int(accHours.Int64)
   378→		}
   379→		stats = append(stats,
   380→			&model.JobsStatistics{
   381→				TotalJobs:      int(jobs.Int64),
   382→				TotalUsers:     int(users.Int64),
   383→				TotalWalltime:  int(walltime.Int64),
   384→				TotalNodeHours: totalNodeHours,
   385→				TotalCoreHours: totalCoreHours,
   386→				TotalAccHours:  totalAccHours,
   387→			})
   388→	}
   389→
   390→	cclog.Debugf("Timer JobStats %s", time.Since(start))
   391→	return stats, nil
   392→}
   393→
   394→// LoadJobStat retrieves a specific statistic for a metric from a job's statistics.
   395→// Returns 0.0 if the metric is not found or statType is invalid.
   396→//
   397→// Parameters:
   398→//   - job: Job struct with populated Statistics field
   399→//   - metric: Name of the metric to query (e.g., "cpu_load", "mem_used")
   400→//   - statType: Type of statistic: "avg", "min", or "max"
   401→//
   402→// Returns the requested statistic value or 0.0 if not found.
   403→func LoadJobStat(job *schema.Job, metric string, statType string) float64 {
   404→	if stats, ok := job.Statistics[metric]; ok {
   405→		switch statType {
   406→		case "avg":
   407→			return stats.Avg
   408→		case "max":
   409→			return stats.Max
   410→		case "min":
   411→			return stats.Min
   412→		default:
   413→			cclog.Errorf("Unknown stat type %s", statType)
   414→		}
   415→	}
   416→
   417→	return 0.0
   418→}
   419→
   420→// JobCountGrouped counts jobs grouped by a dimension without computing detailed statistics.
   421→//
   422→// This is a lightweight alternative to JobsStatsGrouped when only job counts are needed,
   423→// avoiding the overhead of calculating walltime and resource usage metrics.
   424→//
   425→// Parameters:
   426→//   - ctx: Context for security checks
   427→//   - filter: Filters to apply
   428→//   - groupBy: Grouping dimension (User, Project, Cluster, or SubCluster)
   429→//
   430→// Returns JobsStatistics with only ID and TotalJobs populated for each group.
   431→func (r *JobRepository) JobCountGrouped(
   432→	ctx context.Context,
   433→	filter []*model.JobFilter,
   434→	groupBy *model.Aggregate,
   435→) ([]*model.JobsStatistics, error) {
   436→	start := time.Now()
   437→	col := groupBy2column[*groupBy]
   438→	query := r.buildCountQuery(filter, "", col)
   439→	query, err := SecurityCheck(ctx, query)
   440→	if err != nil {
   441→		return nil, err
   442→	}
   443→	rows, err := query.RunWith(r.DB).QueryContext(ctx)
   444→	if err != nil {
   445→		cclog.Warn("Error while querying DB for job statistics")
   446→		return nil, err
   447→	}
   448→	defer rows.Close()
   449→
   450→	stats := make([]*model.JobsStatistics, 0, 100)
   451→
   452→	for rows.Next() {
   453→		var id sql.NullString
   454→		var cnt sql.NullInt64
   455→		if err := rows.Scan(&id, &cnt); err != nil {
   456→			cclog.Warn("Error while scanning rows")
   457→			return nil, err
   458→		}
   459→		if id.Valid {
   460→			stats = append(stats,
   461→				&model.JobsStatistics{
   462→					ID:        id.String,
   463→					TotalJobs: int(cnt.Int64),
   464→				})
   465→		}
   466→	}
   467→
   468→	if err := rows.Err(); err != nil {
   469→		return nil, err
   470→	}
   471→
   472→	cclog.Debugf("Timer JobCountGrouped %s", time.Since(start))
   473→	return stats, nil
   474→}
   475→
   476→// AddJobCountGrouped augments existing statistics with additional job counts by category.
   477→//
   478→// This method enriches JobsStatistics returned by JobsStatsGrouped or JobCountGrouped
   479→// with counts of running or short-running (based on ShortRunningJobsDuration) jobs, matched by group ID.
   480→//
   481→// Parameters:
   482→//   - ctx: Context for security checks
   483→//   - filter: Filters to apply
   484→//   - groupBy: Grouping dimension (must match the dimension used for stats parameter)
   485→//   - stats: Existing statistics to augment (modified in-place by ID matching)
   486→//   - kind: "running" to add RunningJobs count, "short" to add ShortJobs count
   487→//
   488→// Returns the same stats slice with RunningJobs or ShortJobs fields populated per group.
   489→// Groups without matching jobs will have 0 for the added field.
   490→func (r *JobRepository) AddJobCountGrouped(
   491→	ctx context.Context,
   492→	filter []*model.JobFilter,
   493→	groupBy *model.Aggregate,
   494→	stats []*model.JobsStatistics,
   495→	kind string,
   496→) ([]*model.JobsStatistics, error) {
   497→	start := time.Now()
   498→	col := groupBy2column[*groupBy]
   499→	query := r.buildCountQuery(filter, kind, col)
   500→	query, err := SecurityCheck(ctx, query)
   501→	if err != nil {
   502→		return nil, err
   503→	}
   504→	rows, err := query.RunWith(r.DB).QueryContext(ctx)
   505→	if err != nil {
   506→		cclog.Warn("Error while querying DB for job statistics")
   507→		return nil, err
   508→	}
   509→	defer rows.Close()
   510→
   511→	counts := make(map[string]int)
   512→
   513→	for rows.Next() {
   514→		var id sql.NullString
   515→		var cnt sql.NullInt64
   516→		if err := rows.Scan(&id, &cnt); err != nil {
   517→			cclog.Warn("Error while scanning rows")
   518→			return nil, err
   519→		}
   520→		if id.Valid {
   521→			counts[id.String] = int(cnt.Int64)
   522→		}
   523→	}
   524→
   525→	if err := rows.Err(); err != nil {
   526→		return nil, err
   527→	}
   528→
   529→	switch kind {
   530→	case "running":
   531→		for _, s := range stats {
   532→			s.RunningJobs = counts[s.ID]
   533→		}
   534→	case "short":
   535→		for _, s := range stats {
   536→			s.ShortJobs = counts[s.ID]
   537→		}
   538→	}
   539→
   540→	cclog.Debugf("Timer AddJobCountGrouped %s", time.Since(start))
   541→	return stats, nil
   542→}
   543→
   544→// AddJobCount augments existing overall statistics with additional job counts by category.
   545→//
   546→// Similar to AddJobCountGrouped but for ungrouped statistics. Applies the same count
   547→// to all statistics entries (typically just one).
   548→//
   549→// Parameters:
   550→//   - ctx: Context for security checks
   551→//   - filter: Filters to apply
   552→//   - stats: Existing statistics to augment (modified in-place)
   553→//   - kind: "running" to add RunningJobs count, "short" to add ShortJobs count
   554→//
   555→// Returns the same stats slice with RunningJobs or ShortJobs fields set to the total count.
   556→func (r *JobRepository) AddJobCount(
   557→	ctx context.Context,
   558→	filter []*model.JobFilter,
   559→	stats []*model.JobsStatistics,
   560→	kind string,
   561→) ([]*model.JobsStatistics, error) {
   562→	start := time.Now()
   563→	query := r.buildCountQuery(filter, kind, "")
   564→	query, err := SecurityCheck(ctx, query)
   565→	if err != nil {
   566→		return nil, err
   567→	}
   568→	var cnt sql.NullInt64
   569→	if err := query.RunWith(r.DB).QueryRowContext(ctx).Scan(&cnt); err != nil {
   570→		cclog.Warn("Error while querying DB for job count")
   571→		return nil, err
   572→	}
   573→
   574→	count := int(cnt.Int64)
   575→
   576→	switch kind {
   577→	case "running":
   578→		for _, s := range stats {
   579→			s.RunningJobs = count
   580→		}
   581→	case "short":
   582→		for _, s := range stats {
   583→			s.ShortJobs = count
   584→		}
   585→	}
   586→
   587→	cclog.Debugf("Timer AddJobCount %s", time.Since(start))
   588→	return stats, nil
   589→}
   590→
   591→// AddHistograms augments statistics with distribution histograms for job properties.
   592→//
   593→// Generates histogram data for visualization of job duration, node count, core count,
   594→// and accelerator count distributions. Duration histogram uses intelligent binning based
   595→// on the requested resolution.
   596→//
   597→// Parameters:
   598→//   - ctx: Context for security checks
   599→//   - filter: Filters to apply to jobs included in histograms
   600→//   - stat: Statistics struct to augment (modified in-place)
   601→//   - durationBins: Bin size - "1m", "10m", "1h", "6h", "12h", or "24h" (default)
   602→//
   603→// Populates these fields in stat:
   604→//   - HistDuration: Job duration distribution (zero-padded bins)
   605→//   - HistNumNodes: Node count distribution
   606→//   - HistNumCores: Core (hwthread) count distribution
   607→//   - HistNumAccs: Accelerator count distribution
   608→//
   609→// Duration bins are pre-initialized with zeros to ensure consistent ranges for visualization.
   610→// Bin size determines both the width and maximum duration displayed (e.g., "1h" = 48 bins × 1h = 48h max).
   611→func (r *JobRepository) AddHistograms(
   612→	ctx context.Context,
   613→	filter []*model.JobFilter,
   614→	stat *model.JobsStatistics,
   615→	durationBins *string,
   616→) (*model.JobsStatistics, error) {
   617→	start := time.Now()
   618→
   619→	var targetBinCount int
   620→	var targetBinSize int
   621→	switch *durationBins {
   622→	case "1m": // 1 Minute Bins + Max 60 Bins -> Max 60 Minutes
   623→		targetBinCount = 60
   624→		targetBinSize = 60
   625→	case "10m": // 10 Minute Bins + Max 72 Bins -> Max 12 Hours
   626→		targetBinCount = 72
   627→		targetBinSize = 600
   628→	case "1h": // 1 Hour Bins + Max 48 Bins -> Max 48 Hours
   629→		targetBinCount = 48
   630→		targetBinSize = 3600
   631→	case "6h": // 6 Hour Bins + Max 12 Bins -> Max 3 Days
   632→		targetBinCount = 12
   633→		targetBinSize = 21600
   634→	case "12h": // 12 hour Bins + Max 14 Bins -> Max 7 Days
   635→		targetBinCount = 14
   636→		targetBinSize = 43200
   637→	default: // 24h
   638→		targetBinCount = 24
   639→		targetBinSize = 3600
   640→	}
   641→
   642→	var err error
   643→	// Return X-Values always as seconds, will be formatted into minutes and hours in frontend
   644→	value := fmt.Sprintf(`CAST(ROUND(((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) / %d) + 1) as int) as value`, time.Now().Unix(), targetBinSize)
   645→	stat.HistDuration, err = r.jobsDurationStatisticsHistogram(ctx, value, filter, targetBinSize, &targetBinCount)
   646→	if err != nil {
   647→		cclog.Warn("Error while loading job statistics histogram: job duration")
   648→		return nil, err
   649→	}
   650→
   651→	stat.HistNumNodes, err = r.jobsStatisticsHistogram(ctx, "job.num_nodes as value", filter)
   652→	if err != nil {
   653→		cclog.Warn("Error while loading job statistics histogram: num nodes")
   654→		return nil, err
   655→	}
   656→
   657→	stat.HistNumCores, err = r.jobsStatisticsHistogram(ctx, "job.num_hwthreads as value", filter)
   658→	if err != nil {
   659→		cclog.Warn("Error while loading job statistics histogram: num hwthreads")
   660→		return nil, err
   661→	}
   662→
   663→	stat.HistNumAccs, err = r.jobsStatisticsHistogram(ctx, "job.num_acc as value", filter)
   664→	if err != nil {
   665→		cclog.Warn("Error while loading job statistics histogram: num acc")
   666→		return nil, err
   667→	}
   668→
   669→	cclog.Debugf("Timer AddHistograms %s", time.Since(start))
   670→	return stat, nil
   671→}
   672→
   673→// AddMetricHistograms augments statistics with distribution histograms for job metrics.
   674→//
   675→// Generates histogram data for metrics like CPU load, memory usage, etc. Handles running
   676→// and completed jobs differently: running jobs load data from metric backend, completed jobs
   677→// use footprint data from database.
   678→//
   679→// Parameters:
   680→//   - ctx: Context for security checks
   681→//   - filter: Filters to apply (MUST contain State filter for running jobs)
   682→//   - metrics: List of metric names to histogram (e.g., ["cpu_load", "mem_used"])
   683→//   - stat: Statistics struct to augment (modified in-place)
   684→//   - targetBinCount: Number of histogram bins (default: 10)
   685→//
   686→// Populates HistMetrics field in stat with MetricHistoPoints for each metric.
   687→//
   688→// Binning algorithm:
   689→//   - Values normalized to metric's peak value from cluster configuration
   690→//   - Bins evenly distributed from 0 to peak
   691→//   - Pre-initialized with zeros for consistent visualization
   692→//
   693→// Limitations:
   694→//   - Running jobs: Limited to 5000 jobs for performance
   695→//   - Requires valid cluster configuration with metric peak values
   696→//   - Uses footprint statistic (avg/max/min) configured per metric
   697→func (r *JobRepository) AddMetricHistograms(
   698→	ctx context.Context,
   699→	filter []*model.JobFilter,
   700→	metrics []string,
   701→	stat *model.JobsStatistics,
   702→	targetBinCount *int,
   703→) (*model.JobsStatistics, error) {
   704→	start := time.Now()
   705→
   706→	// Running Jobs Only: First query jobdata from sqlite, then query data and make bins
   707→	for _, f := range filter {
   708→		if f.State != nil {
   709→			if len(f.State) == 1 && f.State[0] == "running" {
   710→				stat.HistMetrics = r.runningJobsMetricStatisticsHistogram(ctx, metrics, filter, targetBinCount)
   711→				cclog.Debugf("Timer AddMetricHistograms %s", time.Since(start))
   712→				return stat, nil
   713→			}
   714→		}
   715→	}
   716→
   717→	// All other cases: Query and make bins in sqlite directly
   718→	for _, m := range metrics {
   719→		metricHisto, err := r.jobsMetricStatisticsHistogram(ctx, m, filter, targetBinCount)
   720→		if err != nil {
   721→			cclog.Warnf("Error while loading job metric statistics histogram: %s", m)
   722→			continue
   723→		}
   724→		stat.HistMetrics = append(stat.HistMetrics, metricHisto)
   725→	}
   726→
   727→	cclog.Debugf("Timer AddMetricHistograms %s", time.Since(start))
   728→	return stat, nil
   729→}
   730→
   731→// jobsStatisticsHistogram generates a simple histogram by grouping on a column value.
   732→//
   733→// Used for histograms where the column value directly represents the bin (e.g., node count, core count).
   734→// Unlike duration/metric histograms, this doesn't pre-initialize bins with zeros.
   735→//
   736→// Parameters:
   737→//   - value: SQL expression that produces the histogram value, aliased as "value"
   738→//   - filters: Job filters to apply
   739→//
   740→// Returns histogram points with Value (from column) and Count (number of jobs).
   741→func (r *JobRepository) jobsStatisticsHistogram(
   742→	ctx context.Context,
   743→	value string,
   744→	filters []*model.JobFilter,
   745→) ([]*model.HistoPoint, error) {
   746→	start := time.Now()
   747→	query, qerr := SecurityCheck(ctx,
   748→		sq.Select(value, "COUNT(job.id) AS count").From("job"))
   749→
   750→	if qerr != nil {
   751→		return nil, qerr
   752→	}
   753→
   754→	for _, f := range filters {
   755→		query = BuildWhereClause(f, query)
   756→	}
   757→
   758→	rows, err := query.GroupBy("value").RunWith(r.DB).QueryContext(ctx)
   759→	if err != nil {
   760→		cclog.Error("Error while running query")
   761→		return nil, err
   762→	}
   763→	defer rows.Close()
   764→
   765→	points := make([]*model.HistoPoint, 0)
   766→	// is it possible to introduce zero values here? requires info about bincount
   767→	for rows.Next() {
   768→		point := model.HistoPoint{}
   769→		if err := rows.Scan(&point.Value, &point.Count); err != nil {
   770→			cclog.Warn("Error while scanning rows")
   771→			return nil, err
   772→		}
   773→
   774→		points = append(points, &point)
   775→	}
   776→
   777→	if err := rows.Err(); err != nil {
   778→		return nil, err
   779→	}
   780→
   781→	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
   782→	return points, nil
   783→}
   784→
   785→// jobsDurationStatisticsHistogram generates a duration histogram with pre-initialized bins.
   786→//
   787→// Bins are zero-padded to provide consistent ranges for visualization, unlike simple
   788→// histograms which only return bins with data. The value parameter should compute
   789→// the bin number from job duration.
   790→//
   791→// Parameters:
   792→//   - value: SQL expression computing bin number from duration, aliased as "value"
   793→//   - filters: Job filters to apply
   794→//   - binSizeSeconds: Width of each bin in seconds
   795→//   - targetBinCount: Number of bins to pre-initialize
   796→//
   797→// Returns histogram points with Value (bin_number × binSizeSeconds) and Count.
   798→// All bins from 1 to targetBinCount are returned, with Count=0 for empty bins.
   799→//
   800→// Algorithm:
   801→//  1. Pre-initialize targetBinCount bins with zero counts
   802→//  2. Query database for actual counts per bin
   803→//  3. Match query results to pre-initialized bins by value
   804→//  4. Bins without matches remain at zero
   805→func (r *JobRepository) jobsDurationStatisticsHistogram(
   806→	ctx context.Context,
   807→	value string,
   808→	filters []*model.JobFilter,
   809→	binSizeSeconds int,
   810→	targetBinCount *int,
   811→) ([]*model.HistoPoint, error) {
   812→	start := time.Now()
   813→	query, qerr := SecurityCheck(ctx,
   814→		sq.Select(value, "COUNT(job.id) AS count").From("job"))
   815→
   816→	if qerr != nil {
   817→		return nil, qerr
   818→	}
   819→
   820→	// Each bin represents a duration range: bin N = [N*binSizeSeconds, (N+1)*binSizeSeconds)
   821→	// Example: binSizeSeconds=3600 (1 hour), bin 1 = 0-1h, bin 2 = 1-2h, etc.
   822→	points := make([]*model.HistoPoint, 0)
   823→	for i := 1; i <= *targetBinCount; i++ {
   824→		point := model.HistoPoint{Value: i * binSizeSeconds, Count: 0}
   825→		points = append(points, &point)
   826→	}
   827→
   828→	for _, f := range filters {
   829→		query = BuildWhereClause(f, query)
   830→	}
   831→
   832→	rows, err := query.GroupBy("value").RunWith(r.DB).QueryContext(ctx)
   833→	if err != nil {
   834→		cclog.Error("Error while running query")
   835→		return nil, err
   836→	}
   837→	defer rows.Close()
   838→
   839→	// Match query results to pre-initialized bins.
   840→	// point.Value from query is the bin number; multiply by binSizeSeconds to match bin.Value.
   841→	for rows.Next() {
   842→		point := model.HistoPoint{}
   843→		if err := rows.Scan(&point.Value, &point.Count); err != nil {
   844→			cclog.Warn("Error while scanning rows")
   845→			return nil, err
   846→		}
   847→
   848→		for _, e := range points {
   849→			if e.Value == (point.Value * binSizeSeconds) {
   850→				e.Count = point.Count
   851→				break
   852→			}
   853→		}
   854→	}
   855→
   856→	if err := rows.Err(); err != nil {
   857→		return nil, err
   858→	}
   859→
   860→	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
   861→	return points, nil
   862→}
   863→
   864→// jobsMetricStatisticsHistogram generates a metric histogram using footprint data from completed jobs.
   865→//
   866→// Values are normalized to the metric's peak value and distributed into bins. The algorithm
   867→// is based on SQL histogram generation techniques, extracting metric values from JSON footprint
   868→// and computing bin assignments in SQL.
   869→//
   870→// Parameters:
   871→//   - metric: Metric name (e.g., "cpu_load", "mem_used")
   872→//   - filters: Job filters to apply
   873→//   - bins: Number of bins to generate
   874→//
   875→// Returns MetricHistoPoints with metric name, unit, footprint stat type, and binned data.
   876→//
   877→// Algorithm:
   878→//  1. Determine peak value from cluster configuration (filtered cluster or max across all)
   879→//  2. Generate SQL that extracts footprint value, normalizes to [0,1], multiplies by bin count
   880→//  3. Pre-initialize bins with min/max ranges based on peak value
   881→//  4. Query database for counts per bin
   882→//  5. Match results to pre-initialized bins
   883→//
   884→// Special handling: Values exactly equal to peak are forced into the last bin by multiplying
   885→// peak by 0.999999999 to avoid creating an extra bin.
   886→func (r *JobRepository) jobsMetricStatisticsHistogram(
   887→	ctx context.Context,
   888→	metric string,
   889→	filters []*model.JobFilter,
   890→	bins *int,
   891→) (*model.MetricHistoPoints, error) {
   892→	// Peak value defines the upper bound for binning: values are distributed across
   893→	// bins from 0 to peak. First try to get peak from filtered cluster, otherwise
   894→	// scan all clusters to find the maximum peak value.
   895→	var metricConfig *schema.MetricConfig
   896→	var peak float64
   897→	var unit string
   898→	var footprintStat string
   899→
   900→	// Try to get metric config from filtered cluster
   901→	for _, f := range filters {
   902→		if f.Cluster != nil {
   903→			metricConfig = archive.GetMetricConfig(*f.Cluster.Eq, metric)
   904→			peak = metricConfig.Peak
   905→			unit = metricConfig.Unit.Prefix + metricConfig.Unit.Base
   906→			footprintStat = metricConfig.Footprint
   907→			cclog.Debugf("Cluster %s filter found with peak %f for %s", *f.Cluster.Eq, peak, metric)
   908→		}
   909→	}
   910→
   911→	// If no cluster filter or peak not found, find largest peak across all clusters
   912→	// This ensures histogram can accommodate all possible values
   913→	if peak == 0.0 {
   914→		for _, c := range archive.Clusters {
   915→			for _, m := range c.MetricConfig {
   916→				if m.Name == metric {
   917→					if m.Peak > peak {
   918→						peak = m.Peak
   919→					}
   920→					if unit == "" {
   921→						unit = m.Unit.Prefix + m.Unit.Base
   922→					}
   923→					if footprintStat == "" {
   924→						footprintStat = m.Footprint
   925→					}
   926→				}
   927→			}
   928→		}
   929→	}
   930→
   931→	// Construct SQL histogram bins using normalized values.
   932→	// Algorithm based on: https://jereze.com/code/sql-histogram/ (modified)
   933→	start := time.Now()
   934→
   935→	// Bin calculation formula:
   936→	//   bin_number = CAST( (value / peak) * num_bins AS INTEGER ) + 1
   937→	// Special case: value == peak would create bin N+1, so we test for equality
   938→	// and multiply peak by 0.999999999 to force it into bin N.
   939→	binQuery := fmt.Sprintf(`CAST(
   940→		((case when json_extract(footprint, "$.%s") = %f then %f*0.999999999 else json_extract(footprint, "$.%s") end) / %f)
   941→		* %v as INTEGER )`,
   942→		(metric + "_" + footprintStat), peak, peak, (metric + "_" + footprintStat), peak, *bins)
   943→
   944→	mainQuery := sq.Select(
   945→		fmt.Sprintf(`%s + 1 as bin`, binQuery),
   946→		`count(*) as count`,
   947→	).From("job").Where(
   948→		"JSON_VALID(footprint)",
   949→	).Where(fmt.Sprintf(`json_extract(footprint, "$.%s") is not null and json_extract(footprint, "$.%s") <= %f`, (metric + "_" + footprintStat), (metric + "_" + footprintStat), peak))
   950→
   951→	mainQuery, qerr := SecurityCheck(ctx, mainQuery)
   952→	if qerr != nil {
   953→		return nil, qerr
   954→	}
   955→
   956→	for _, f := range filters {
   957→		mainQuery = BuildWhereClause(f, mainQuery)
   958→	}
   959→
   960→	mainQuery = mainQuery.GroupBy("bin").OrderBy("bin")
   961→
   962→	rows, err := mainQuery.RunWith(r.DB).QueryContext(ctx)
   963→	if err != nil {
   964→		cclog.Errorf("Error while running mainQuery: %s", err)
   965→		return nil, err
   966→	}
   967→	defer rows.Close()
   968→
   969→	// Pre-initialize bins with calculated min/max ranges.
   970→	// Example: peak=1000, bins=10 -> bin 1=[0,100), bin 2=[100,200), ..., bin 10=[900,1000]
   971→	points := make([]*model.MetricHistoPoint, 0)
   972→	binStep := int(peak) / *bins
   973→	for i := 1; i <= *bins; i++ {
   974→		binMin := (binStep * (i - 1))
   975→		binMax := (binStep * i)
   976→		epoint := model.MetricHistoPoint{Bin: &i, Count: 0, Min: &binMin, Max: &binMax}
   977→		points = append(points, &epoint)
   978→	}
   979→
   980→	// Match query results to pre-initialized bins.
   981→	for rows.Next() {
   982→		rpoint := model.MetricHistoPoint{}
   983→		if err := rows.Scan(&rpoint.Bin, &rpoint.Count); err != nil {
   984→			cclog.Warnf("Error while scanning rows for %s", metric)
   985→			return nil, err
   986→		}
   987→
   988→		for _, e := range points {
   989→			if e.Bin != nil && rpoint.Bin != nil && *e.Bin == *rpoint.Bin {
   990→				e.Count = rpoint.Count
   991→				break
   992→			}
   993→		}
   994→	}
   995→
   996→	if err := rows.Err(); err != nil {
   997→		return nil, err
   998→	}
   999→
  1000→	result := model.MetricHistoPoints{Metric: metric, Unit: unit, Stat: &footprintStat, Data: points}
  1001→
  1002→	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
  1003→	return &result, nil
  1004→}
  1005→
  1006→// runningJobsMetricStatisticsHistogram generates metric histograms for running jobs using live data.
  1007→//
  1008→// Unlike completed jobs which use footprint data from the database, running jobs require
  1009→// fetching current metric averages from the metric backend (via metricdispatch).
  1010→//
  1011→// Parameters:
  1012→//   - metrics: List of metric names
  1013→//   - filters: Job filters (should filter to running jobs only)
  1014→//   - bins: Number of histogram bins
  1015→//
  1016→// Returns slice of MetricHistoPoints, one per metric.
  1017→//
  1018→// Limitations:
  1019→//   - Maximum 5000 jobs (returns nil if more jobs match)
  1020→//   - Requires metric backend availability
  1021→//   - Bins based on metric peak values from cluster configuration
  1022→//
  1023→// Algorithm:
  1024→//  1. Query first 5001 jobs to check count limit
  1025→//  2. Load metric averages for all jobs via metricdispatch
  1026→//  3. For each metric, create bins based on peak value
  1027→//  4. Iterate averages and count jobs per bin
  1028→func (r *JobRepository) runningJobsMetricStatisticsHistogram(
  1029→	ctx context.Context,
  1030→	metrics []string,
  1031→	filters []*model.JobFilter,
  1032→	bins *int,
  1033→) []*model.MetricHistoPoints {
  1034→	// Get Jobs
  1035→	jobs, err := r.QueryJobs(ctx, filters, &model.PageRequest{Page: 1, ItemsPerPage: 5000 + 1}, nil)
  1036→	if err != nil {
  1037→		cclog.Errorf("Error while querying jobs for footprint: %s", err)
  1038→		return nil
  1039→	}
  1040→	if len(jobs) > 5000 {
  1041→		cclog.Errorf("too many jobs matched (max: %d)", 5000)
  1042→		return nil
  1043→	}
  1044→
  1045→	// Get AVGs from metric repo
  1046→	avgs := make([][]schema.Float, len(metrics))
  1047→	for i := range avgs {
  1048→		avgs[i] = make([]schema.Float, 0, len(jobs))
  1049→	}
  1050→
  1051→	for _, job := range jobs {
  1052→		if job.MonitoringStatus == schema.MonitoringStatusDisabled || job.MonitoringStatus == schema.MonitoringStatusArchivingFailed {
  1053→			continue
  1054→		}
  1055→
  1056→		if err := metricdispatch.LoadAverages(job, metrics, avgs, ctx); err != nil {
  1057→			cclog.Errorf("Error while loading averages for histogram: %s", err)
  1058→			return nil
  1059→		}
  1060→	}
  1061→
  1062→	// Iterate metrics to fill endresult
  1063→	data := make([]*model.MetricHistoPoints, 0)
  1064→	for idx, metric := range metrics {
  1065→		// Get specific Peak or largest Peak
  1066→		var metricConfig *schema.MetricConfig
  1067→		var peak float64
  1068→		var unit string
  1069→
  1070→		for _, f := range filters {
  1071→			if f.Cluster != nil {
  1072→				metricConfig = archive.GetMetricConfig(*f.Cluster.Eq, metric)
  1073→				peak = metricConfig.Peak
  1074→				unit = metricConfig.Unit.Prefix + metricConfig.Unit.Base
  1075→			}
  1076→		}
  1077→
  1078→		if peak == 0.0 {
  1079→			for _, c := range archive.Clusters {
  1080→				for _, m := range c.MetricConfig {
  1081→					if m.Name == metric {
  1082→						if m.Peak > peak {
  1083→							peak = m.Peak
  1084→						}
  1085→						if unit == "" {
  1086→							unit = m.Unit.Prefix + m.Unit.Base
  1087→						}
  1088→					}
  1089→				}
  1090→			}
  1091→		}
  1092→
  1093→		// Make and fill bins
  1094→		peakBin := int(peak) / *bins
  1095→
  1096→		points := make([]*model.MetricHistoPoint, 0)
  1097→		for b := 0; b < *bins; b++ {
  1098→			count := 0
  1099→			bindex := b + 1
  1100→			bmin := peakBin * b
  1101→			bmax := peakBin * (b + 1)
  1102→
  1103→			// Iterate AVG values for indexed metric and count for bins
  1104→			for _, val := range avgs[idx] {
  1105→				if int(val) >= bmin && int(val) < bmax {
  1106→					count += 1
  1107→				}
  1108→			}
  1109→
  1110→			// Append Bin to Metric Result Array
  1111→			point := model.MetricHistoPoint{Bin: &bindex, Count: count, Min: &bmin, Max: &bmax}
  1112→			points = append(points, &point)
  1113→		}
  1114→
  1115→		// Append Metric Result Array to final results array
  1116→		result := model.MetricHistoPoints{Metric: metric, Unit: unit, Data: points}
  1117→		data = append(data, &result)
  1118→	}
  1119→
  1120→	return data
  1121→}
  1122→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

// This file contains job statistics and histogram generation functionality for the JobRepository.
//
// # Job Statistics
//
// The statistics methods provide aggregated metrics about jobs including total jobs, users,
// walltime, and resource usage (nodes, cores, accelerators). Statistics can be computed:
//   - Overall (JobsStats): Single aggregate across all matching jobs
//   - Grouped (JobsStatsGrouped): Aggregated by user, project, cluster, or subcluster
//   - Counts (JobCountGrouped, AddJobCount): Simple job counts with optional filtering
//
// All statistics methods support filtering via JobFilter and respect security contexts.
//
// # Histograms
//
// Histogram methods generate distribution data for visualization:
//   - Duration, nodes, cores, accelerators (AddHistograms)
//   - Job metrics like CPU load, memory usage (AddMetricHistograms)
//
// Histograms use intelligent binning:
//   - Duration: Variable bin sizes (1m, 10m, 1h, 6h, 12h, 24h) with zero-padding
//   - Resources: Natural value-based bins
//   - Metrics: Normalized to peak values with configurable bin counts
//
// # Running vs. Completed Jobs
//
// Statistics handle running jobs specially:
//   - Duration calculated as (now - start_time) for running jobs
//   - Metric histograms for running jobs load data from metric backend instead of footprint
//   - Job state filtering distinguishes running/completed jobs
//
// # Performance Considerations
//
// - All queries use prepared statements via stmtCache
// - Complex aggregations use SQL for efficiency
// - Histogram pre-initialization ensures consistent bin ranges
// - Metric histogram queries limited to 5000 jobs for running job analysis

package repository

import (
	"context"
	"database/sql"
	"fmt"
	"time"

	"github.com/ClusterCockpit/cc-backend/internal/config"
	"github.com/ClusterCockpit/cc-backend/internal/graph/model"
	"github.com/ClusterCockpit/cc-backend/internal/metricdispatch"
	"github.com/ClusterCockpit/cc-backend/pkg/archive"
	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/schema"
	sq "github.com/Masterminds/squirrel"
)

// groupBy2column maps GraphQL Aggregate enum values to their corresponding database column names.
// Used by JobsStatsGrouped and JobCountGrouped to translate user-facing grouping dimensions
// into SQL GROUP BY clauses. GraphQL validation ensures only valid enum values are accepted.
var groupBy2column = map[model.Aggregate]string{
	model.AggregateUser:       "job.hpc_user",
	model.AggregateProject:    "job.project",
	model.AggregateCluster:    "job.cluster",
	model.AggregateSubcluster: "job.subcluster",
}

// sortBy2column maps GraphQL SortByAggregate enum values to their corresponding computed column names.
// Used by JobsStatsGrouped to translate sort preferences into SQL ORDER BY clauses.
// Column names match the AS aliases used in buildStatsQuery.
var sortBy2column = map[model.SortByAggregate]string{
	model.SortByAggregateTotaljobs:      "totalJobs",
	model.SortByAggregateTotalusers:     "totalUsers",
	model.SortByAggregateTotalwalltime:  "totalWalltime",
	model.SortByAggregateTotalnodes:     "totalNodes",
	model.SortByAggregateTotalnodehours: "totalNodeHours",
	model.SortByAggregateTotalcores:     "totalCores",
	model.SortByAggregateTotalcorehours: "totalCoreHours",
	model.SortByAggregateTotalaccs:      "totalAccs",
	model.SortByAggregateTotalacchours:  "totalAccHours",
}

// buildCountQuery constructs a SQL query to count jobs with optional grouping and filtering.
//
// Parameters:
//   - filter: Job filters to apply (cluster, user, time range, etc.)
//   - kind: Special filter - "running" for running jobs only, "short" for jobs under threshold
//   - col: Column name to GROUP BY; empty string for total count without grouping
//
// Returns a SelectBuilder that produces either:
//   - Single count: COUNT(job.id) when col is empty
//   - Grouped counts: col, COUNT(job.id) when col is specified
//
// The kind parameter enables counting specific job categories:
//   - "running": Only jobs with job_state = 'running'
//   - "short": Only jobs with duration < ShortRunningJobsDuration config value
//   - empty: All jobs matching filters
func (r *JobRepository) buildCountQuery(
	filter []*model.JobFilter,
	kind string,
	col string,
) sq.SelectBuilder {
	var query sq.SelectBuilder

	if col != "" {
		query = sq.Select(col, "COUNT(job.id)").From("job").GroupBy(col)
	} else {
		query = sq.Select("COUNT(job.id)").From("job")
	}

	switch kind {
	case "running":
		query = query.Where("job.job_state = ?", "running")
	case "short":
		query = query.Where("job.duration < ?", config.Keys.ShortRunningJobsDuration)
	}

	for _, f := range filter {
		query = BuildWhereClause(f, query)
	}

	return query
}

// buildStatsQuery constructs a SQL query to compute comprehensive job statistics with optional grouping.
//
// Parameters:
//   - filter: Job filters to apply (cluster, user, time range, etc.)
//   - col: Column name to GROUP BY; empty string for overall statistics without grouping
//
// Returns a SelectBuilder that produces comprehensive statistics:
//   - totalJobs: Count of jobs
//   - totalUsers: Count of distinct users (always 0 when grouping by user)
//   - totalWalltime: Sum of job durations in hours
//   - totalNodes: Sum of nodes used across all jobs
//   - totalNodeHours: Sum of (duration × num_nodes) in hours
//   - totalCores: Sum of hardware threads used across all jobs
//   - totalCoreHours: Sum of (duration × num_hwthreads) in hours
//   - totalAccs: Sum of accelerators used across all jobs
//   - totalAccHours: Sum of (duration × num_acc) in hours
//
// Special handling:
//   - Running jobs: Duration calculated as (now - start_time) instead of stored duration
//   - Grouped queries: Also select grouping column and user's display name from hpc_user table
//   - All time values converted from seconds to hours (÷ 3600) and rounded
func (r *JobRepository) buildStatsQuery(
	filter []*model.JobFilter,
	col string,
) sq.SelectBuilder {
	var query sq.SelectBuilder

	if col != "" {
		query = sq.Select(
			col,
			"name",
			"COUNT(job.id) as totalJobs",
			"COUNT(DISTINCT job.hpc_user) AS totalUsers",
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END)) / 3600) as int) as totalWalltime`, time.Now().Unix()),
			`CAST(SUM(job.num_nodes) as int) as totalNodes`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_nodes) / 3600) as int) as totalNodeHours`, time.Now().Unix()),
			`CAST(SUM(job.num_hwthreads) as int) as totalCores`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_hwthreads) / 3600) as int) as totalCoreHours`, time.Now().Unix()),
			`CAST(SUM(job.num_acc) as int) as totalAccs`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_acc) / 3600) as int) as totalAccHours`, time.Now().Unix()),
		).From("job").LeftJoin("hpc_user ON hpc_user.username = job.hpc_user").GroupBy(col)
	} else {
		query = sq.Select(
			"COUNT(job.id) as totalJobs",
			"COUNT(DISTINCT job.hpc_user) AS totalUsers",
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END)) / 3600) as int)`, time.Now().Unix()),
			`CAST(SUM(job.num_nodes) as int)`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_nodes) / 3600) as int)`, time.Now().Unix()),
			`CAST(SUM(job.num_hwthreads) as int)`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_hwthreads) / 3600) as int)`, time.Now().Unix()),
			`CAST(SUM(job.num_acc) as int)`,
			fmt.Sprintf(`CAST(ROUND(SUM((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) * job.num_acc) / 3600) as int)`, time.Now().Unix()),
		).From("job")
	}

	for _, f := range filter {
		query = BuildWhereClause(f, query)
	}

	return query
}

// JobsStatsGrouped computes comprehensive job statistics grouped by a dimension (user, project, cluster, or subcluster).
//
// This is the primary method for generating aggregated statistics views in the UI, providing
// metrics like total jobs, walltime, and resource usage broken down by the specified grouping.
//
// Parameters:
//   - ctx: Context for security checks and cancellation
//   - filter: Filters to apply (time range, cluster, job state, etc.)
//   - page: Optional pagination (ItemsPerPage: -1 disables pagination)
//   - sortBy: Optional sort column (totalJobs, totalWalltime, totalCoreHours, etc.)
//   - groupBy: Required grouping dimension (User, Project, Cluster, or SubCluster)
//
// Returns a slice of JobsStatistics, one per group, with:
//   - ID: The group identifier (username, project name, cluster name, etc.)
//   - Name: Display name (for users, from hpc_user.name; empty for other groups)
//   - Statistics: totalJobs, totalUsers, totalWalltime, resource usage metrics
//
// Security: Respects user roles via SecurityCheck - users see only their own data unless admin/support.
// Performance: Results are sorted in SQL and pagination applied before scanning rows.
func (r *JobRepository) JobsStatsGrouped(
	ctx context.Context,
	filter []*model.JobFilter,
	page *model.PageRequest,
	sortBy *model.SortByAggregate,
	groupBy *model.Aggregate,
) ([]*model.JobsStatistics, error) {
	start := time.Now()
	col := groupBy2column[*groupBy]
	query := r.buildStatsQuery(filter, col)

	query, err := SecurityCheck(ctx, query)
	if err != nil {
		return nil, err
	}

	if sortBy != nil {
		sortBy := sortBy2column[*sortBy]
		query = query.OrderBy(fmt.Sprintf("%s DESC", sortBy))
	}
	if page != nil && page.ItemsPerPage != -1 {
		limit := uint64(page.ItemsPerPage)
		query = query.Offset((uint64(page.Page) - 1) * limit).Limit(limit)
	}

	rows, err := query.RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Warn("Error while querying DB for job statistics")
		return nil, err
	}
	defer rows.Close()

	stats := make([]*model.JobsStatistics, 0, 100)

	for rows.Next() {
		var id sql.NullString
		var name sql.NullString
		var jobs, users, walltime, nodes, nodeHours, cores, coreHours, accs, accHours sql.NullInt64
		if err := rows.Scan(&id, &name, &jobs, &users, &walltime, &nodes, &nodeHours, &cores, &coreHours, &accs, &accHours); err != nil {
			cclog.Warnf("Error while scanning rows: %s", err.Error())
			return nil, err
		}

		if id.Valid {
			var totalJobs, totalUsers, totalWalltime, totalNodes, totalNodeHours, totalCores, totalCoreHours, totalAccs, totalAccHours int
			var personName string

			if name.Valid {
				personName = name.String
			}

			if jobs.Valid {
				totalJobs = int(jobs.Int64)
			}

			if users.Valid {
				totalUsers = int(users.Int64)
			}

			if walltime.Valid {
				totalWalltime = int(walltime.Int64)
			}

			if nodes.Valid {
				totalNodes = int(nodes.Int64)
			}
			if cores.Valid {
				totalCores = int(cores.Int64)
			}
			if accs.Valid {
				totalAccs = int(accs.Int64)
			}

			if nodeHours.Valid {
				totalNodeHours = int(nodeHours.Int64)
			}
			if coreHours.Valid {
				totalCoreHours = int(coreHours.Int64)
			}
			if accHours.Valid {
				totalAccHours = int(accHours.Int64)
			}

			if col == "job.hpc_user" {
				// name := r.getUserName(ctx, id.String)
				stats = append(stats,
					&model.JobsStatistics{
						ID:             id.String,
						Name:           personName,
						TotalJobs:      totalJobs,
						TotalWalltime:  totalWalltime,
						TotalNodes:     totalNodes,
						TotalNodeHours: totalNodeHours,
						TotalCores:     totalCores,
						TotalCoreHours: totalCoreHours,
						TotalAccs:      totalAccs,
						TotalAccHours:  totalAccHours,
					})
			} else {
				stats = append(stats,
					&model.JobsStatistics{
						ID:             id.String,
						TotalJobs:      totalJobs,
						TotalUsers:     totalUsers,
						TotalWalltime:  totalWalltime,
						TotalNodes:     totalNodes,
						TotalNodeHours: totalNodeHours,
						TotalCores:     totalCores,
						TotalCoreHours: totalCoreHours,
						TotalAccs:      totalAccs,
						TotalAccHours:  totalAccHours,
					})
			}
		}
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	cclog.Debugf("Timer JobsStatsGrouped %s", time.Since(start))
	return stats, nil
}

// JobsStats computes overall job statistics across all matching jobs without grouping.
//
// This method provides a single aggregate view of job metrics, useful for dashboard
// summaries and overall system utilization reports.
//
// Parameters:
//   - ctx: Context for security checks and cancellation
//   - filter: Filters to apply (time range, cluster, job state, etc.)
//
// Returns a single-element slice containing aggregate statistics:
//   - totalJobs, totalUsers, totalWalltime
//   - totalNodeHours, totalCoreHours, totalAccHours
//
// Unlike JobsStatsGrouped, this returns overall totals without breaking down by dimension.
// Security checks are applied via SecurityCheck to respect user access levels.
func (r *JobRepository) JobsStats(
	ctx context.Context,
	filter []*model.JobFilter,
) ([]*model.JobsStatistics, error) {
	start := time.Now()
	query := r.buildStatsQuery(filter, "")
	query, err := SecurityCheck(ctx, query)
	if err != nil {
		return nil, err
	}

	row := query.RunWith(r.DB).QueryRowContext(ctx)
	stats := make([]*model.JobsStatistics, 0, 1)

	var jobs, users, walltime, nodes, nodeHours, cores, coreHours, accs, accHours sql.NullInt64
	if err := row.Scan(&jobs, &users, &walltime, &nodes, &nodeHours, &cores, &coreHours, &accs, &accHours); err != nil {
		cclog.Warn("Error while scanning rows")
		return nil, err
	}

	if jobs.Valid {
		var totalNodeHours, totalCoreHours, totalAccHours int

		if nodeHours.Valid {
			totalNodeHours = int(nodeHours.Int64)
		}
		if coreHours.Valid {
			totalCoreHours = int(coreHours.Int64)
		}
		if accHours.Valid {
			totalAccHours = int(accHours.Int64)
		}
		stats = append(stats,
			&model.JobsStatistics{
				TotalJobs:      int(jobs.Int64),
				TotalUsers:     int(users.Int64),
				TotalWalltime:  int(walltime.Int64),
				TotalNodeHours: totalNodeHours,
				TotalCoreHours: totalCoreHours,
				TotalAccHours:  totalAccHours,
			})
	}

	cclog.Debugf("Timer JobStats %s", time.Since(start))
	return stats, nil
}

// LoadJobStat retrieves a specific statistic for a metric from a job's statistics.
// Returns 0.0 if the metric is not found or statType is invalid.
//
// Parameters:
//   - job: Job struct with populated Statistics field
//   - metric: Name of the metric to query (e.g., "cpu_load", "mem_used")
//   - statType: Type of statistic: "avg", "min", or "max"
//
// Returns the requested statistic value or 0.0 if not found.
func LoadJobStat(job *schema.Job, metric string, statType string) float64 {
	if stats, ok := job.Statistics[metric]; ok {
		switch statType {
		case "avg":
			return stats.Avg
		case "max":
			return stats.Max
		case "min":
			return stats.Min
		default:
			cclog.Errorf("Unknown stat type %s", statType)
		}
	}

	return 0.0
}

// JobCountGrouped counts jobs grouped by a dimension without computing detailed statistics.
//
// This is a lightweight alternative to JobsStatsGrouped when only job counts are needed,
// avoiding the overhead of calculating walltime and resource usage metrics.
//
// Parameters:
//   - ctx: Context for security checks
//   - filter: Filters to apply
//   - groupBy: Grouping dimension (User, Project, Cluster, or SubCluster)
//
// Returns JobsStatistics with only ID and TotalJobs populated for each group.
func (r *JobRepository) JobCountGrouped(
	ctx context.Context,
	filter []*model.JobFilter,
	groupBy *model.Aggregate,
) ([]*model.JobsStatistics, error) {
	start := time.Now()
	col := groupBy2column[*groupBy]
	query := r.buildCountQuery(filter, "", col)
	query, err := SecurityCheck(ctx, query)
	if err != nil {
		return nil, err
	}
	rows, err := query.RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Warn("Error while querying DB for job statistics")
		return nil, err
	}
	defer rows.Close()

	stats := make([]*model.JobsStatistics, 0, 100)

	for rows.Next() {
		var id sql.NullString
		var cnt sql.NullInt64
		if err := rows.Scan(&id, &cnt); err != nil {
			cclog.Warn("Error while scanning rows")
			return nil, err
		}
		if id.Valid {
			stats = append(stats,
				&model.JobsStatistics{
					ID:        id.String,
					TotalJobs: int(cnt.Int64),
				})
		}
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	cclog.Debugf("Timer JobCountGrouped %s", time.Since(start))
	return stats, nil
}

// AddJobCountGrouped augments existing statistics with additional job counts by category.
//
// This method enriches JobsStatistics returned by JobsStatsGrouped or JobCountGrouped
// with counts of running or short-running (based on ShortRunningJobsDuration) jobs, matched by group ID.
//
// Parameters:
//   - ctx: Context for security checks
//   - filter: Filters to apply
//   - groupBy: Grouping dimension (must match the dimension used for stats parameter)
//   - stats: Existing statistics to augment (modified in-place by ID matching)
//   - kind: "running" to add RunningJobs count, "short" to add ShortJobs count
//
// Returns the same stats slice with RunningJobs or ShortJobs fields populated per group.
// Groups without matching jobs will have 0 for the added field.
func (r *JobRepository) AddJobCountGrouped(
	ctx context.Context,
	filter []*model.JobFilter,
	groupBy *model.Aggregate,
	stats []*model.JobsStatistics,
	kind string,
) ([]*model.JobsStatistics, error) {
	start := time.Now()
	col := groupBy2column[*groupBy]
	query := r.buildCountQuery(filter, kind, col)
	query, err := SecurityCheck(ctx, query)
	if err != nil {
		return nil, err
	}
	rows, err := query.RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Warn("Error while querying DB for job statistics")
		return nil, err
	}
	defer rows.Close()

	counts := make(map[string]int)

	for rows.Next() {
		var id sql.NullString
		var cnt sql.NullInt64
		if err := rows.Scan(&id, &cnt); err != nil {
			cclog.Warn("Error while scanning rows")
			return nil, err
		}
		if id.Valid {
			counts[id.String] = int(cnt.Int64)
		}
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	switch kind {
	case "running":
		for _, s := range stats {
			s.RunningJobs = counts[s.ID]
		}
	case "short":
		for _, s := range stats {
			s.ShortJobs = counts[s.ID]
		}
	}

	cclog.Debugf("Timer AddJobCountGrouped %s", time.Since(start))
	return stats, nil
}

// AddJobCount augments existing overall statistics with additional job counts by category.
//
// Similar to AddJobCountGrouped but for ungrouped statistics. Applies the same count
// to all statistics entries (typically just one).
//
// Parameters:
//   - ctx: Context for security checks
//   - filter: Filters to apply
//   - stats: Existing statistics to augment (modified in-place)
//   - kind: "running" to add RunningJobs count, "short" to add ShortJobs count
//
// Returns the same stats slice with RunningJobs or ShortJobs fields set to the total count.
func (r *JobRepository) AddJobCount(
	ctx context.Context,
	filter []*model.JobFilter,
	stats []*model.JobsStatistics,
	kind string,
) ([]*model.JobsStatistics, error) {
	start := time.Now()
	query := r.buildCountQuery(filter, kind, "")
	query, err := SecurityCheck(ctx, query)
	if err != nil {
		return nil, err
	}
	var cnt sql.NullInt64
	if err := query.RunWith(r.DB).QueryRowContext(ctx).Scan(&cnt); err != nil {
		cclog.Warn("Error while querying DB for job count")
		return nil, err
	}

	count := int(cnt.Int64)

	switch kind {
	case "running":
		for _, s := range stats {
			s.RunningJobs = count
		}
	case "short":
		for _, s := range stats {
			s.ShortJobs = count
		}
	}

	cclog.Debugf("Timer AddJobCount %s", time.Since(start))
	return stats, nil
}

// AddHistograms augments statistics with distribution histograms for job properties.
//
// Generates histogram data for visualization of job duration, node count, core count,
// and accelerator count distributions. Duration histogram uses intelligent binning based
// on the requested resolution.
//
// Parameters:
//   - ctx: Context for security checks
//   - filter: Filters to apply to jobs included in histograms
//   - stat: Statistics struct to augment (modified in-place)
//   - durationBins: Bin size - "1m", "10m", "1h", "6h", "12h", or "24h" (default)
//
// Populates these fields in stat:
//   - HistDuration: Job duration distribution (zero-padded bins)
//   - HistNumNodes: Node count distribution
//   - HistNumCores: Core (hwthread) count distribution
//   - HistNumAccs: Accelerator count distribution
//
// Duration bins are pre-initialized with zeros to ensure consistent ranges for visualization.
// Bin size determines both the width and maximum duration displayed (e.g., "1h" = 48 bins × 1h = 48h max).
func (r *JobRepository) AddHistograms(
	ctx context.Context,
	filter []*model.JobFilter,
	stat *model.JobsStatistics,
	durationBins *string,
) (*model.JobsStatistics, error) {
	start := time.Now()

	var targetBinCount int
	var targetBinSize int
	switch *durationBins {
	case "1m": // 1 Minute Bins + Max 60 Bins -> Max 60 Minutes
		targetBinCount = 60
		targetBinSize = 60
	case "10m": // 10 Minute Bins + Max 72 Bins -> Max 12 Hours
		targetBinCount = 72
		targetBinSize = 600
	case "1h": // 1 Hour Bins + Max 48 Bins -> Max 48 Hours
		targetBinCount = 48
		targetBinSize = 3600
	case "6h": // 6 Hour Bins + Max 12 Bins -> Max 3 Days
		targetBinCount = 12
		targetBinSize = 21600
	case "12h": // 12 hour Bins + Max 14 Bins -> Max 7 Days
		targetBinCount = 14
		targetBinSize = 43200
	default: // 24h
		targetBinCount = 24
		targetBinSize = 3600
	}

	var err error
	// Return X-Values always as seconds, will be formatted into minutes and hours in frontend
	value := fmt.Sprintf(`CAST(ROUND(((CASE WHEN job.job_state = "running" THEN %d - job.start_time ELSE job.duration END) / %d) + 1) as int) as value`, time.Now().Unix(), targetBinSize)
	stat.HistDuration, err = r.jobsDurationStatisticsHistogram(ctx, value, filter, targetBinSize, &targetBinCount)
	if err != nil {
		cclog.Warn("Error while loading job statistics histogram: job duration")
		return nil, err
	}

	stat.HistNumNodes, err = r.jobsStatisticsHistogram(ctx, "job.num_nodes as value", filter)
	if err != nil {
		cclog.Warn("Error while loading job statistics histogram: num nodes")
		return nil, err
	}

	stat.HistNumCores, err = r.jobsStatisticsHistogram(ctx, "job.num_hwthreads as value", filter)
	if err != nil {
		cclog.Warn("Error while loading job statistics histogram: num hwthreads")
		return nil, err
	}

	stat.HistNumAccs, err = r.jobsStatisticsHistogram(ctx, "job.num_acc as value", filter)
	if err != nil {
		cclog.Warn("Error while loading job statistics histogram: num acc")
		return nil, err
	}

	cclog.Debugf("Timer AddHistograms %s", time.Since(start))
	return stat, nil
}

// AddMetricHistograms augments statistics with distribution histograms for job metrics.
//
// Generates histogram data for metrics like CPU load, memory usage, etc. Handles running
// and completed jobs differently: running jobs load data from metric backend, completed jobs
// use footprint data from database.
//
// Parameters:
//   - ctx: Context for security checks
//   - filter: Filters to apply (MUST contain State filter for running jobs)
//   - metrics: List of metric names to histogram (e.g., ["cpu_load", "mem_used"])
//   - stat: Statistics struct to augment (modified in-place)
//   - targetBinCount: Number of histogram bins (default: 10)
//
// Populates HistMetrics field in stat with MetricHistoPoints for each metric.
//
// Binning algorithm:
//   - Values normalized to metric's peak value from cluster configuration
//   - Bins evenly distributed from 0 to peak
//   - Pre-initialized with zeros for consistent visualization
//
// Limitations:
//   - Running jobs: Limited to 5000 jobs for performance
//   - Requires valid cluster configuration with metric peak values
//   - Uses footprint statistic (avg/max/min) configured per metric
func (r *JobRepository) AddMetricHistograms(
	ctx context.Context,
	filter []*model.JobFilter,
	metrics []string,
	stat *model.JobsStatistics,
	targetBinCount *int,
) (*model.JobsStatistics, error) {
	start := time.Now()

	// Running Jobs Only: First query jobdata from sqlite, then query data and make bins
	for _, f := range filter {
		if f.State != nil {
			if len(f.State) == 1 && f.State[0] == "running" {
				stat.HistMetrics = r.runningJobsMetricStatisticsHistogram(ctx, metrics, filter, targetBinCount)
				cclog.Debugf("Timer AddMetricHistograms %s", time.Since(start))
				return stat, nil
			}
		}
	}

	// All other cases: Query and make bins in sqlite directly
	for _, m := range metrics {
		metricHisto, err := r.jobsMetricStatisticsHistogram(ctx, m, filter, targetBinCount)
		if err != nil {
			cclog.Warnf("Error while loading job metric statistics histogram: %s", m)
			continue
		}
		stat.HistMetrics = append(stat.HistMetrics, metricHisto)
	}

	cclog.Debugf("Timer AddMetricHistograms %s", time.Since(start))
	return stat, nil
}

// jobsStatisticsHistogram generates a simple histogram by grouping on a column value.
//
// Used for histograms where the column value directly represents the bin (e.g., node count, core count).
// Unlike duration/metric histograms, this doesn't pre-initialize bins with zeros.
//
// Parameters:
//   - value: SQL expression that produces the histogram value, aliased as "value"
//   - filters: Job filters to apply
//
// Returns histogram points with Value (from column) and Count (number of jobs).
func (r *JobRepository) jobsStatisticsHistogram(
	ctx context.Context,
	value string,
	filters []*model.JobFilter,
) ([]*model.HistoPoint, error) {
	start := time.Now()
	query, qerr := SecurityCheck(ctx,
		sq.Select(value, "COUNT(job.id) AS count").From("job"))

	if qerr != nil {
		return nil, qerr
	}

	for _, f := range filters {
		query = BuildWhereClause(f, query)
	}

	rows, err := query.GroupBy("value").RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Error("Error while running query")
		return nil, err
	}
	defer rows.Close()

	points := make([]*model.HistoPoint, 0)
	// is it possible to introduce zero values here? requires info about bincount
	for rows.Next() {
		point := model.HistoPoint{}
		if err := rows.Scan(&point.Value, &point.Count); err != nil {
			cclog.Warn("Error while scanning rows")
			return nil, err
		}

		points = append(points, &point)
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
	return points, nil
}

// jobsDurationStatisticsHistogram generates a duration histogram with pre-initialized bins.
//
// Bins are zero-padded to provide consistent ranges for visualization, unlike simple
// histograms which only return bins with data. The value parameter should compute
// the bin number from job duration.
//
// Parameters:
//   - value: SQL expression computing bin number from duration, aliased as "value"
//   - filters: Job filters to apply
//   - binSizeSeconds: Width of each bin in seconds
//   - targetBinCount: Number of bins to pre-initialize
//
// Returns histogram points with Value (bin_number × binSizeSeconds) and Count.
// All bins from 1 to targetBinCount are returned, with Count=0 for empty bins.
//
// Algorithm:
//  1. Pre-initialize targetBinCount bins with zero counts
//  2. Query database for actual counts per bin
//  3. Match query results to pre-initialized bins by value
//  4. Bins without matches remain at zero
func (r *JobRepository) jobsDurationStatisticsHistogram(
	ctx context.Context,
	value string,
	filters []*model.JobFilter,
	binSizeSeconds int,
	targetBinCount *int,
) ([]*model.HistoPoint, error) {
	start := time.Now()
	query, qerr := SecurityCheck(ctx,
		sq.Select(value, "COUNT(job.id) AS count").From("job"))

	if qerr != nil {
		return nil, qerr
	}

	// Each bin represents a duration range: bin N = [N*binSizeSeconds, (N+1)*binSizeSeconds)
	// Example: binSizeSeconds=3600 (1 hour), bin 1 = 0-1h, bin 2 = 1-2h, etc.
	points := make([]*model.HistoPoint, 0)
	for i := 1; i <= *targetBinCount; i++ {
		point := model.HistoPoint{Value: i * binSizeSeconds, Count: 0}
		points = append(points, &point)
	}

	for _, f := range filters {
		query = BuildWhereClause(f, query)
	}

	rows, err := query.GroupBy("value").RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Error("Error while running query")
		return nil, err
	}
	defer rows.Close()

	// Match query results to pre-initialized bins.
	// point.Value from query is the bin number; multiply by binSizeSeconds to match bin.Value.
	for rows.Next() {
		point := model.HistoPoint{}
		if err := rows.Scan(&point.Value, &point.Count); err != nil {
			cclog.Warn("Error while scanning rows")
			return nil, err
		}

		for _, e := range points {
			if e.Value == (point.Value * binSizeSeconds) {
				e.Count = point.Count
				break
			}
		}
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
	return points, nil
}

// jobsMetricStatisticsHistogram generates a metric histogram using footprint data from completed jobs.
//
// Values are normalized to the metric's peak value and distributed into bins. The algorithm
// is based on SQL histogram generation techniques, extracting metric values from JSON footprint
// and computing bin assignments in SQL.
//
// Parameters:
//   - metric: Metric name (e.g., "cpu_load", "mem_used")
//   - filters: Job filters to apply
//   - bins: Number of bins to generate
//
// Returns MetricHistoPoints with metric name, unit, footprint stat type, and binned data.
//
// Algorithm:
//  1. Determine peak value from cluster configuration (filtered cluster or max across all)
//  2. Generate SQL that extracts footprint value, normalizes to [0,1], multiplies by bin count
//  3. Pre-initialize bins with min/max ranges based on peak value
//  4. Query database for counts per bin
//  5. Match results to pre-initialized bins
//
// Special handling: Values exactly equal to peak are forced into the last bin by multiplying
// peak by 0.999999999 to avoid creating an extra bin.
func (r *JobRepository) jobsMetricStatisticsHistogram(
	ctx context.Context,
	metric string,
	filters []*model.JobFilter,
	bins *int,
) (*model.MetricHistoPoints, error) {
	// Peak value defines the upper bound for binning: values are distributed across
	// bins from 0 to peak. First try to get peak from filtered cluster, otherwise
	// scan all clusters to find the maximum peak value.
	var metricConfig *schema.MetricConfig
	var peak float64
	var unit string
	var footprintStat string

	// Try to get metric config from filtered cluster
	for _, f := range filters {
		if f.Cluster != nil {
			metricConfig = archive.GetMetricConfig(*f.Cluster.Eq, metric)
			peak = metricConfig.Peak
			unit = metricConfig.Unit.Prefix + metricConfig.Unit.Base
			footprintStat = metricConfig.Footprint
			cclog.Debugf("Cluster %s filter found with peak %f for %s", *f.Cluster.Eq, peak, metric)
		}
	}

	// If no cluster filter or peak not found, find largest peak across all clusters
	// This ensures histogram can accommodate all possible values
	if peak == 0.0 {
		for _, c := range archive.Clusters {
			for _, m := range c.MetricConfig {
				if m.Name == metric {
					if m.Peak > peak {
						peak = m.Peak
					}
					if unit == "" {
						unit = m.Unit.Prefix + m.Unit.Base
					}
					if footprintStat == "" {
						footprintStat = m.Footprint
					}
				}
			}
		}
	}

	// Construct SQL histogram bins using normalized values.
	// Algorithm based on: https://jereze.com/code/sql-histogram/ (modified)
	start := time.Now()

	// Bin calculation formula:
	//   bin_number = CAST( (value / peak) * num_bins AS INTEGER ) + 1
	// Special case: value == peak would create bin N+1, so we test for equality
	// and multiply peak by 0.999999999 to force it into bin N.
	binQuery := fmt.Sprintf(`CAST(
		((case when json_extract(footprint, "$.%s") = %f then %f*0.999999999 else json_extract(footprint, "$.%s") end) / %f)
		* %v as INTEGER )`,
		(metric + "_" + footprintStat), peak, peak, (metric + "_" + footprintStat), peak, *bins)

	mainQuery := sq.Select(
		fmt.Sprintf(`%s + 1 as bin`, binQuery),
		`count(*) as count`,
	).From("job").Where(
		"JSON_VALID(footprint)",
	).Where(fmt.Sprintf(`json_extract(footprint, "$.%s") is not null and json_extract(footprint, "$.%s") <= %f`, (metric + "_" + footprintStat), (metric + "_" + footprintStat), peak))

	mainQuery, qerr := SecurityCheck(ctx, mainQuery)
	if qerr != nil {
		return nil, qerr
	}

	for _, f := range filters {
		mainQuery = BuildWhereClause(f, mainQuery)
	}

	mainQuery = mainQuery.GroupBy("bin").OrderBy("bin")

	rows, err := mainQuery.RunWith(r.DB).QueryContext(ctx)
	if err != nil {
		cclog.Errorf("Error while running mainQuery: %s", err)
		return nil, err
	}
	defer rows.Close()

	// Pre-initialize bins with calculated min/max ranges.
	// Example: peak=1000, bins=10 -> bin 1=[0,100), bin 2=[100,200), ..., bin 10=[900,1000]
	points := make([]*model.MetricHistoPoint, 0)
	binStep := int(peak) / *bins
	for i := 1; i <= *bins; i++ {
		binMin := (binStep * (i - 1))
		binMax := (binStep * i)
		epoint := model.MetricHistoPoint{Bin: &i, Count: 0, Min: &binMin, Max: &binMax}
		points = append(points, &epoint)
	}

	// Match query results to pre-initialized bins.
	for rows.Next() {
		rpoint := model.MetricHistoPoint{}
		if err := rows.Scan(&rpoint.Bin, &rpoint.Count); err != nil {
			cclog.Warnf("Error while scanning rows for %s", metric)
			return nil, err
		}

		for _, e := range points {
			if e.Bin != nil && rpoint.Bin != nil && *e.Bin == *rpoint.Bin {
				e.Count = rpoint.Count
				break
			}
		}
	}

	if err := rows.Err(); err != nil {
		return nil, err
	}

	result := model.MetricHistoPoints{Metric: metric, Unit: unit, Stat: &footprintStat, Data: points}

	cclog.Debugf("Timer jobsStatisticsHistogram %s", time.Since(start))
	return &result, nil
}

// runningJobsMetricStatisticsHistogram generates metric histograms for running jobs using live data.
//
// Unlike completed jobs which use footprint data from the database, running jobs require
// fetching current metric averages from the metric backend (via metricdispatch).
//
// Parameters:
//   - metrics: List of metric names
//   - filters: Job filters (should filter to running jobs only)
//   - bins: Number of histogram bins
//
// Returns slice of MetricHistoPoints, one per metric.
//
// Limitations:
//   - Maximum 5000 jobs (returns nil if more jobs match)
//   - Requires metric backend availability
//   - Bins based on metric peak values from cluster configuration
//
// Algorithm:
//  1. Query first 5001 jobs to check count limit
//  2. Load metric averages for all jobs via metricdispatch
//  3. For each metric, create bins based on peak value
//  4. Iterate averages and count jobs per bin
func (r *JobRepository) runningJobsMetricStatisticsHistogram(
	ctx context.Context,
	metrics []string,
	filters []*model.JobFilter,
	bins *int,
) []*model.MetricHistoPoints {
	// Get Jobs
	jobs, err := r.QueryJobs(ctx, filters, &model.PageRequest{Page: 1, ItemsPerPage: 5000 + 1}, nil)
	if err != nil {
		cclog.Errorf("Error while querying jobs for footprint: %s", err)
		return nil
	}
	if len(jobs) > 5000 {
		cclog.Errorf("too many jobs matched (max: %d)", 5000)
		return nil
	}

	// Get AVGs from metric repo
	avgs := make([][]schema.Float, len(metrics))
	for i := range avgs {
		avgs[i] = make([]schema.Float, 0, len(jobs))
	}

	for _, job := range jobs {
		if job.MonitoringStatus == schema.MonitoringStatusDisabled || job.MonitoringStatus == schema.MonitoringStatusArchivingFailed {
			continue
		}

		if err := metricdispatch.LoadAverages(job, metrics, avgs, ctx); err != nil {
			cclog.Errorf("Error while loading averages for histogram: %s", err)
			return nil
		}
	}

	// Iterate metrics to fill endresult
	data := make([]*model.MetricHistoPoints, 0)
	for idx, metric := range metrics {
		// Get specific Peak or largest Peak
		var metricConfig *schema.MetricConfig
		var peak float64
		var unit string

		for _, f := range filters {
			if f.Cluster != nil {
				metricConfig = archive.GetMetricConfig(*f.Cluster.Eq, metric)
				peak = metricConfig.Peak
				unit = metricConfig.Unit.Prefix + metricConfig.Unit.Base
			}
		}

		if peak == 0.0 {
			for _, c := range archive.Clusters {
				for _, m := range c.MetricConfig {
					if m.Name == metric {
						if m.Peak > peak {
							peak = m.Peak
						}
						if unit == "" {
							unit = m.Unit.Prefix + m.Unit.Base
						}
					}
				}
			}
		}

		// Make and fill bins
		peakBin := int(peak) / *bins

		points := make([]*model.MetricHistoPoint, 0)
		for b := 0; b < *bins; b++ {
			count := 0
			bindex := b + 1
			bmin := peakBin * b
			bmax := peakBin * (b + 1)

			// Iterate AVG values for indexed metric and count for bins
			for _, val := range avgs[idx] {
				if int(val) >= bmin && int(val) < bmax {
					count += 1
				}
			}

			// Append Bin to Metric Result Array
			point := model.MetricHistoPoint{Bin: &bindex, Count: count, Min: &bmin, Max: &bmax}
			points = append(points, &point)
		}

		// Append Metric Result Array to final results array
		result := model.MetricHistoPoints{Metric: metric, Unit: unit, Data: points}
		data = append(data, &result)
	}

	return data
}


   640→		hasNextPage = len(nextJobs) == 1
   641→	}
   642→
   643→	return &model.JobResultList{Items: jobs, Count: &count, HasNextPage: &hasNextPage}, nil
   644→}
   645→
   646→// JobsStatistics is the resolver for the jobsStatistics field.
   647→func (r *queryResolver) JobsStatistics(ctx context.Context, filter []*model.JobFilter, metrics []string, page *model.PageRequest, sortBy *model.SortByAggregate, groupBy *model.Aggregate, numDurationBins *string, numMetricBins *int) ([]*model.JobsStatistics, error) {
   648→	var err error
   649→	var stats []*model.JobsStatistics
   650→
   651→	// Top Level Defaults
   652→	defaultDurationBins := "1h"
   653→	defaultMetricBins := 10
   654→
   655→	if requireField(ctx, "totalJobs") || requireField(ctx, "totalUsers") || requireField(ctx, "totalWalltime") || requireField(ctx, "totalNodes") || requireField(ctx, "totalCores") ||
   656→		requireField(ctx, "totalAccs") || requireField(ctx, "totalNodeHours") || requireField(ctx, "totalCoreHours") || requireField(ctx, "totalAccHours") {
   657→		if groupBy == nil {
   658→			stats, err = r.Repo.JobsStats(ctx, filter)
   659→		} else {
   660→			stats, err = r.Repo.JobsStatsGrouped(ctx, filter, page, sortBy, groupBy)
   661→		}
   662→	} else {
   663→		stats = make([]*model.JobsStatistics, 0, 1)
   664→		stats = append(stats, &model.JobsStatistics{})
   665→	}
   666→
   667→	if groupBy != nil {
   668→		if requireField(ctx, "shortJobs") {
   669→			stats, err = r.Repo.AddJobCountGrouped(ctx, filter, groupBy, stats, "short")
   670→		}
   671→		if requireField(ctx, "runningJobs") {
   672→			stats, err = r.Repo.AddJobCountGrouped(ctx, filter, groupBy, stats, "running")
   673→		}
   674→	} else {
   675→		if requireField(ctx, "shortJobs") {
   676→			stats, err = r.Repo.AddJobCount(ctx, filter, stats, "short")
   677→		}
   678→		if requireField(ctx, "runningJobs") {
   679→			stats, err = r.Repo.AddJobCount(ctx, filter, stats, "running")
   680→		}
   681→	}
   682→
   683→	if err != nil {
   684→		return nil, err
   685→	}
   686→
   687→	if requireField(ctx, "histDuration") || requireField(ctx, "histNumNodes") || requireField(ctx, "histNumCores") || requireField(ctx, "histNumAccs") {
   688→
   689→		if numDurationBins == nil {
   690→			numDurationBins = &defaultDurationBins
   691→		}
   692→
   693→		if groupBy == nil {
   694→			stats[0], err = r.Repo.AddHistograms(ctx, filter, stats[0], numDurationBins)
   695→			if err != nil {
   696→				return nil, err
   697→			}
   698→		} else {
   699→			return nil, errors.New("histograms only implemented without groupBy argument")
   700→		}
   701→	}
   702→
   703→	if requireField(ctx, "histMetrics") {
   704→
   705→		if numMetricBins == nil {
   706→			numMetricBins = &defaultMetricBins
   707→		}
   708→
   709→		if groupBy == nil {
   710→			stats[0], err = r.Repo.AddMetricHistograms(ctx, filter, metrics, stats[0], numMetricBins)
   711→			if err != nil {
   712→				return nil, err
   713→			}
   714→		} else {
   715→			return nil, errors.New("metric histograms only implemented without groupBy argument")
   716→		}
   717→	}
   718→
   719→	return stats, nil
   720→}
   721→
   722→// JobsMetricStats is the resolver for the jobsMetricStats field.
   723→func (r *queryResolver) JobsMetricStats(ctx context.Context, filter []*model.JobFilter, metrics []string) ([]*model.JobStats, error) {
   724→	// No Paging, Fixed Order by StartTime ASC
   725→	order := &model.OrderByInput{
   726→		Field: "startTime",
   727→		Type:  "col",
   728→		Order: "ASC",
   729→	}
   730→
   731→	jobs, err := r.Repo.QueryJobs(ctx, filter, nil, order)
   732→	if err != nil {
   733→		cclog.Warn("Error while querying jobs for comparison")
   734→		return nil, err
   735→	}
   736→
   737→	res := []*model.JobStats{}
   738→	for _, job := range jobs {
   739→		data, err := metricdispatch.LoadJobStats(job, metrics, ctx)

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

		hasNextPage = len(nextJobs) == 1
	}

	return &model.JobResultList{Items: jobs, Count: &count, HasNextPage: &hasNextPage}, nil
}

// JobsStatistics is the resolver for the jobsStatistics field.
func (r *queryResolver) JobsStatistics(ctx context.Context, filter []*model.JobFilter, metrics []string, page *model.PageRequest, sortBy *model.SortByAggregate, groupBy *model.Aggregate, numDurationBins *string, numMetricBins *int) ([]*model.JobsStatistics, error) {
	var err error
	var stats []*model.JobsStatistics

	// Top Level Defaults
	defaultDurationBins := "1h"
	defaultMetricBins := 10

	if requireField(ctx, "totalJobs") || requireField(ctx, "totalUsers") || requireField(ctx, "totalWalltime") || requireField(ctx, "totalNodes") || requireField(ctx, "totalCores") ||
		requireField(ctx, "totalAccs") || requireField(ctx, "totalNodeHours") || requireField(ctx, "totalCoreHours") || requireField(ctx, "totalAccHours") {
		if groupBy == nil {
			stats, err = r.Repo.JobsStats(ctx, filter)
		} else {
			stats, err = r.Repo.JobsStatsGrouped(ctx, filter, page, sortBy, groupBy)
		}
	} else {
		stats = make([]*model.JobsStatistics, 0, 1)
		stats = append(stats, &model.JobsStatistics{})
	}

	if groupBy != nil {
		if requireField(ctx, "shortJobs") {
			stats, err = r.Repo.AddJobCountGrouped(ctx, filter, groupBy, stats, "short")
		}
		if requireField(ctx, "runningJobs") {
			stats, err = r.Repo.AddJobCountGrouped(ctx, filter, groupBy, stats, "running")
		}
	} else {
		if requireField(ctx, "shortJobs") {
			stats, err = r.Repo.AddJobCount(ctx, filter, stats, "short")
		}
		if requireField(ctx, "runningJobs") {
			stats, err = r.Repo.AddJobCount(ctx, filter, stats, "running")
		}
	}

	if err != nil {
		return nil, err
	}

	if requireField(ctx, "histDuration") || requireField(ctx, "histNumNodes") || requireField(ctx, "histNumCores") || requireField(ctx, "histNumAccs") {

		if numDurationBins == nil {
			numDurationBins = &defaultDurationBins
		}

		if groupBy == nil {
			stats[0], err = r.Repo.AddHistograms(ctx, filter, stats[0], numDurationBins)
			if err != nil {
				return nil, err
			}
		} else {
			return nil, errors.New("histograms only implemented without groupBy argument")
		}
	}

	if requireField(ctx, "histMetrics") {

		if numMetricBins == nil {
			numMetricBins = &defaultMetricBins
		}

		if groupBy == nil {
			stats[0], err = r.Repo.AddMetricHistograms(ctx, filter, metrics, stats[0], numMetricBins)
			if err != nil {
				return nil, err
			}
		} else {
			return nil, errors.New("metric histograms only implemented without groupBy argument")
		}
	}

	return stats, nil
}

// JobsMetricStats is the resolver for the jobsMetricStats field.
func (r *queryResolver) JobsMetricStats(ctx context.Context, filter []*model.JobFilter, metrics []string) ([]*model.JobStats, error) {
	// No Paging, Fixed Order by StartTime ASC
	order := &model.OrderByInput{
		Field: "startTime",
		Type:  "col",
		Order: "ASC",
	}

	jobs, err := r.Repo.QueryJobs(ctx, filter, nil, order)
	if err != nil {
		cclog.Warn("Error while querying jobs for comparison")
		return nil, err
	}

	res := []*model.JobStats{}
	for _, job := range jobs {
		data, err := metricdispatch.LoadJobStats(job, metrics, ctx)


Let me check the model definition to confirm field names:


internal/graph/model/models_gen.go:133:	RunningJobs    int                  `json:"runningJobs"`
internal/graph/model/models_gen.go:134:	ShortJobs      int                  `json:"shortJobs"`


Good. Now I have everything I need. Let me implement all four fixes: