<local-command-caveat>Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.</local-command-caveat>


<command-name>/clear</command-name>
            <command-message>clear</command-message>
            <command-args></command-args>


<local-command-stdout></local-command-stdout>


Compare and analyze the UpdateNodestate REST vs NATS implementations and provide an identical functionality in NATS compared to REST.


Agent

Explore REST UpdateNodeStates

Prompt:
Find and thoroughly analyze the REST implementation for updating node states in the cc-backend project. Look in internal/api/ for the REST handler. I need:
1. The full implementation of the UpdateNodeStates REST endpoint
2. The request format/struct it expects
3. All validation and business logic
4. Database operations performed
5. Any helper functions called

Search in internal/api/rest.go and related files. Also check internal/repository/ for any node-related repository methods used.


Find and thoroughly analyze the REST implementation for updating node states in the cc-backend project. Look in internal/api/ for the REST handler. I need:
1. The full implementation of the UpdateNodeStates REST endpoint
2. The request format/struct it expects
3. All validation and business logic
4. Database operations performed
5. Any helper functions called

Search in internal/api/rest.go and related files. Also check internal/repository/ for any node-related repository methods used.

Prompt:
Find and thoroughly analyze the REST implementation for updating node states in the cc-backend project. Look in internal/api/ for the REST handler. I need:
1. The full implementation of the UpdateNodeStates REST endpoint
2. The request format/struct it expects
3. All validation and business logic
4. Database operations performed
5. Any helper functions called

Search in internal/api/rest.go and related files. Also check internal/repository/ for any node-related repository methods used.


Explore NATS node state handling

Prompt:
Find and thoroughly analyze the NATS implementation for handling node state updates in the cc-backend project. Look in internal/api/nats.go. I need:
1. The full implementation of the NATS node state handler
2. The message format it expects
3. All validation and business logic
4. Database operations performed
5. How it differs from the REST implementation

Also check the config for NATS subjects and any related types/structs.


Find and thoroughly analyze the NATS implementation for handling node state updates in the cc-backend project. Look in internal/api/nats.go. I need:
1. The full implementation of the NATS node state handler
2. The message format it expects
3. All validation and business logic
4. Database operations performed
5. How it differs from the REST implementation

Also check the config for NATS subjects and any related types/structs.

Prompt:
Find and thoroughly analyze the NATS implementation for handling node state updates in the cc-backend project. Look in internal/api/nats.go. I need:
1. The full implementation of the NATS node state handler
2. The message format it expects
3. All validation and business logic
4. Database operations performed
5. How it differs from the REST implementation

Also check the config for NATS subjects and any related types/structs.


Bash

find /Users/jan/prg/ClusterCockpit/cc-backend/internal/api -type f -name "*.go" | head -20


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package api
     7→
     8→import (
     9→	"database/sql"
    10→	"encoding/json"
    11→	"strings"
    12→	"sync"
    13→	"time"
    14→
    15→	"github.com/ClusterCockpit/cc-backend/internal/archiver"
    16→	"github.com/ClusterCockpit/cc-backend/internal/config"
    17→	"github.com/ClusterCockpit/cc-backend/internal/importer"
    18→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    19→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    20→	lp "github.com/ClusterCockpit/cc-lib/v2/ccMessage"
    21→	"github.com/ClusterCockpit/cc-lib/v2/nats"
    22→	"github.com/ClusterCockpit/cc-lib/v2/receivers"
    23→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    24→	influx "github.com/ClusterCockpit/cc-line-protocol/v2/lineprotocol"
    25→)
    26→
    27→// natsMessage wraps a raw NATS message with its subject for channel-based processing.
    28→type natsMessage struct {
    29→	subject string
    30→	data    []byte
    31→}
    32→
    33→// NatsAPI provides NATS subscription-based handlers for Job and Node operations.
    34→// It mirrors the functionality of the REST API but uses NATS messaging with
    35→// InfluxDB line protocol as the message format.
    36→//
    37→// # Message Format
    38→//
    39→// All NATS messages use InfluxDB line protocol format (https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/)
    40→// with the following structure:
    41→//
    42→//	measurement,tag1=value1,tag2=value2 field1=value1,field2=value2 timestamp
    43→//
    44→// # Job Events
    45→//
    46→// Job start/stop events use the "job" measurement with a "function" tag to distinguish operations:
    47→//
    48→//	job,function=start_job event="{...JSON payload...}" <timestamp>
    49→//	job,function=stop_job event="{...JSON payload...}" <timestamp>
    50→//
    51→// The JSON payload in the "event" field follows the schema.Job or StopJobAPIRequest structure.
    52→//
    53→// Example job start message:
    54→//
    55→//	job,function=start_job event="{\"jobId\":1001,\"user\":\"testuser\",\"cluster\":\"testcluster\",...}" 1234567890000000000
    56→//
    57→// # Node State Events
    58→//
    59→// Node state updates use the "nodestate" measurement with cluster information:
    60→//
    61→//	nodestate event="{...JSON payload...}" <timestamp>
    62→//
    63→// The JSON payload follows the UpdateNodeStatesRequest structure.
    64→//
    65→// Example node state message:
    66→//
    67→//	nodestate event="{\"cluster\":\"testcluster\",\"nodes\":[{\"hostname\":\"node01\",\"states\":[\"idle\"]}]}" 1234567890000000000
    68→type NatsAPI struct {
    69→	JobRepository *repository.JobRepository
    70→	// RepositoryMutex protects job creation operations from race conditions
    71→	// when checking for duplicate jobs during startJob calls.
    72→	RepositoryMutex sync.Mutex
    73→	// jobCh receives job event messages for processing by worker goroutines.
    74→	jobCh chan natsMessage
    75→	// nodeCh receives node state messages for processing by worker goroutines.
    76→	nodeCh chan natsMessage
    77→}
    78→
    79→// NewNatsAPI creates a new NatsAPI instance with channel-based worker pools.
    80→// Concurrency is configured via NATSConfig (defaults: JobConcurrency=8, NodeConcurrency=2).
    81→func NewNatsAPI() *NatsAPI {
    82→	jobConc := 8
    83→	nodeConc := 2
    84→
    85→	if s := config.Keys.APISubjects; s != nil {
    86→		if s.JobConcurrency > 0 {
    87→			jobConc = s.JobConcurrency
    88→		}
    89→		if s.NodeConcurrency > 0 {
    90→			nodeConc = s.NodeConcurrency
    91→		}
    92→	}
    93→
    94→	api := &NatsAPI{
    95→		JobRepository: repository.GetJobRepository(),
    96→		jobCh:         make(chan natsMessage, jobConc),
    97→		nodeCh:        make(chan natsMessage, nodeConc),
    98→	}
    99→
   100→	// Start worker goroutines
   101→	for range jobConc {
   102→		go api.jobWorker()
   103→	}
   104→	for range nodeConc {
   105→		go api.nodeWorker()
   106→	}
   107→
   108→	return api
   109→}
   110→
   111→// jobWorker processes job event messages from the job channel.
   112→func (api *NatsAPI) jobWorker() {
   113→	for msg := range api.jobCh {
   114→		api.handleJobEvent(msg.subject, msg.data)
   115→	}
   116→}
   117→
   118→// nodeWorker processes node state messages from the node channel.
   119→func (api *NatsAPI) nodeWorker() {
   120→	for msg := range api.nodeCh {
   121→		api.handleNodeState(msg.subject, msg.data)
   122→	}
   123→}
   124→
   125→// StartSubscriptions registers all NATS subscriptions for Job and Node APIs.
   126→// Messages are delivered to buffered channels and processed by worker goroutines.
   127→// Returns an error if the NATS client is not available or subscription fails.
   128→func (api *NatsAPI) StartSubscriptions() error {
   129→	client := nats.GetClient()
   130→	if client == nil {
   131→		cclog.Warn("NATS client not available, skipping API subscriptions")
   132→		return nil
   133→	}
   134→
   135→	if config.Keys.APISubjects != nil {
   136→		s := config.Keys.APISubjects
   137→
   138→		if err := client.Subscribe(s.SubjectJobEvent, func(subject string, data []byte) {
   139→			api.jobCh <- natsMessage{subject: subject, data: data}
   140→		}); err != nil {
   141→			return err
   142→		}
   143→
   144→		if err := client.Subscribe(s.SubjectNodeState, func(subject string, data []byte) {
   145→			api.nodeCh <- natsMessage{subject: subject, data: data}
   146→		}); err != nil {
   147→			return err
   148→		}
   149→
   150→		cclog.Info("NATS API subscriptions started")
   151→	}
   152→	return nil
   153→}
   154→
   155→// processJobEvent routes job event messages to the appropriate handler based on the "function" tag.
   156→// Validates that required tags and fields are present before processing.
   157→func (api *NatsAPI) processJobEvent(msg lp.CCMessage) {
   158→	function, ok := msg.GetTag("function")
   159→	if !ok {
   160→		cclog.Errorf("Job event is missing required tag 'function': measurement=%s", msg.Name())
   161→		return
   162→	}
   163→
   164→	switch function {
   165→	case "start_job":
   166→		v, ok := msg.GetEventValue()
   167→		if !ok {
   168→			cclog.Errorf("Job start event is missing event field with JSON payload")
   169→			return
   170→		}
   171→		api.handleStartJob(v)
   172→
   173→	case "stop_job":
   174→		v, ok := msg.GetEventValue()
   175→		if !ok {
   176→			cclog.Errorf("Job stop event is missing event field with JSON payload")
   177→			return
   178→		}
   179→		api.handleStopJob(v)
   180→
   181→	default:
   182→		cclog.Warnf("Unknown job event function '%s', expected 'start_job' or 'stop_job'", function)
   183→	}
   184→}
   185→
   186→// handleJobEvent processes job-related messages received via NATS using InfluxDB line protocol.
   187→// The message must be in line protocol format with measurement="job" and include:
   188→//   - tag "function" with value "start_job" or "stop_job"
   189→//   - field "event" containing JSON payload (schema.Job or StopJobAPIRequest)
   190→//
   191→// Example: job,function=start_job event="{\"jobId\":1001,...}" 1234567890000000000
   192→func (api *NatsAPI) handleJobEvent(subject string, data []byte) {
   193→	if len(data) == 0 {
   194→		cclog.Warnf("NATS %s: received empty message", subject)
   195→		return
   196→	}
   197→
   198→	d := influx.NewDecoderWithBytes(data)
   199→
   200→	for d.Next() {
   201→		m, err := receivers.DecodeInfluxMessage(d)
   202→		if err != nil {
   203→			cclog.Errorf("NATS %s: failed to decode InfluxDB line protocol message: %v", subject, err)
   204→			return
   205→		}
   206→
   207→		if !m.IsEvent() {
   208→			cclog.Debugf("NATS %s: received non-event message, skipping", subject)
   209→			continue
   210→		}
   211→
   212→		if m.Name() == "job" {
   213→			api.processJobEvent(m)
   214→		} else {
   215→			cclog.Debugf("NATS %s: unexpected measurement name '%s', expected 'job'", subject, m.Name())
   216→		}
   217→	}
   218→}
   219→
   220→// handleStartJob processes job start messages received via NATS.
   221→// The payload parameter contains JSON following the schema.Job structure.
   222→// Jobs are validated, checked for duplicates, and inserted into the database.
   223→func (api *NatsAPI) handleStartJob(payload string) {
   224→	if payload == "" {
   225→		cclog.Error("NATS start job: payload is empty")
   226→		return
   227→	}
   228→	req := schema.Job{
   229→		Shared:           "none",
   230→		MonitoringStatus: schema.MonitoringStatusRunningOrArchiving,
   231→	}
   232→
   233→	dec := json.NewDecoder(strings.NewReader(payload))
   234→	dec.DisallowUnknownFields()
   235→	if err := dec.Decode(&req); err != nil {
   236→		cclog.Errorf("NATS start job: parsing request failed: %v", err)
   237→		return
   238→	}
   239→
   240→	cclog.Debugf("NATS start job: %s", req.GoString())
   241→	req.State = schema.JobStateRunning
   242→
   243→	if err := importer.SanityChecks(&req); err != nil {
   244→		cclog.Errorf("NATS start job: sanity check failed: %v", err)
   245→		return
   246→	}
   247→
   248→	var unlockOnce sync.Once
   249→	api.RepositoryMutex.Lock()
   250→	defer unlockOnce.Do(api.RepositoryMutex.Unlock)
   251→
   252→	jobs, err := api.JobRepository.FindAll(&req.JobID, &req.Cluster, nil)
   253→	if err != nil && err != sql.ErrNoRows {
   254→		cclog.Errorf("NATS start job: checking for duplicate failed: %v", err)
   255→		return
   256→	}
   257→	if err == nil {
   258→		for _, job := range jobs {
   259→			if (req.StartTime - job.StartTime) < secondsPerDay {
   260→				cclog.Errorf("NATS start job: job with jobId %d, cluster %s already exists (dbid: %d)",
   261→					req.JobID, req.Cluster, job.ID)
   262→				return
   263→			}
   264→		}
   265→	}
   266→
   267→	// When tags are present, insert directly into the job table so that the
   268→	// returned ID can be used with AddTagOrCreate (which queries the job table).
   269→	var id int64
   270→	if len(req.Tags) > 0 {
   271→		id, err = api.JobRepository.StartDirect(&req)
   272→	} else {
   273→		id, err = api.JobRepository.Start(&req)
   274→	}
   275→	if err != nil {
   276→		cclog.Errorf("NATS start job: insert into database failed: %v", err)
   277→		return
   278→	}
   279→	unlockOnce.Do(api.RepositoryMutex.Unlock)
   280→
   281→	for _, tag := range req.Tags {
   282→		if _, err := api.JobRepository.AddTagOrCreate(nil, id, tag.Type, tag.Name, tag.Scope); err != nil {
   283→			cclog.Errorf("NATS start job: adding tag to new job %d failed: %v", id, err)
   284→			return
   285→		}
   286→	}
   287→
   288→	cclog.Infof("NATS: new job (id: %d): cluster=%s, jobId=%d, user=%s, startTime=%d",
   289→		id, req.Cluster, req.JobID, req.User, req.StartTime)
   290→}
   291→
   292→// handleStopJob processes job stop messages received via NATS.
   293→// The payload parameter contains JSON following the StopJobAPIRequest structure.
   294→// The job is marked as stopped in the database and archiving is triggered if monitoring is enabled.
   295→func (api *NatsAPI) handleStopJob(payload string) {
   296→	if payload == "" {
   297→		cclog.Error("NATS stop job: payload is empty")
   298→		return
   299→	}
   300→	var req StopJobAPIRequest
   301→
   302→	dec := json.NewDecoder(strings.NewReader(payload))
   303→	dec.DisallowUnknownFields()
   304→	if err := dec.Decode(&req); err != nil {
   305→		cclog.Errorf("NATS job stop: parsing request failed: %v", err)
   306→		return
   307→	}
   308→
   309→	if req.JobID == nil {
   310→		cclog.Errorf("NATS job stop: the field 'jobId' is required")
   311→		return
   312→	}
   313→
   314→	isCached := false
   315→	job, err := api.JobRepository.FindCached(req.JobID, req.Cluster, req.StartTime)
   316→	if err != nil {
   317→		// Not in cache, try main job table
   318→		job, err = api.JobRepository.Find(req.JobID, req.Cluster, req.StartTime)
   319→		if err != nil {
   320→			cclog.Errorf("NATS job stop: finding job failed: %v", err)
   321→			return
   322→		}
   323→	} else {
   324→		isCached = true
   325→	}
   326→
   327→	if job.State != schema.JobStateRunning {
   328→		cclog.Errorf("NATS job stop: jobId %d (id %d) on %s: job has already been stopped (state is: %s)",
   329→			job.JobID, job.ID, job.Cluster, job.State)
   330→		return
   331→	}
   332→
   333→	if job.StartTime > req.StopTime {
   334→		cclog.Errorf("NATS job stop: jobId %d (id %d) on %s: stopTime %d must be >= startTime %d",
   335→			job.JobID, job.ID, job.Cluster, req.StopTime, job.StartTime)
   336→		return
   337→	}
   338→
   339→	if req.State != "" && !req.State.Valid() {
   340→		cclog.Errorf("NATS job stop: jobId %d (id %d) on %s: invalid job state: %#v",
   341→			job.JobID, job.ID, job.Cluster, req.State)
   342→		return
   343→	} else if req.State == "" {
   344→		req.State = schema.JobStateCompleted
   345→	}
   346→
   347→	job.Duration = int32(req.StopTime - job.StartTime)
   348→	job.State = req.State
   349→	api.JobRepository.Mutex.Lock()
   350→	defer api.JobRepository.Mutex.Unlock()
   351→
   352→	// If the job is still in job_cache, transfer it to the job table first
   353→	if isCached {
   354→		newID, err := api.JobRepository.TransferCachedJobToMain(*job.ID)
   355→		if err != nil {
   356→			cclog.Errorf("NATS job stop: jobId %d (id %d) on %s: transferring cached job failed: %v",
   357→				job.JobID, *job.ID, job.Cluster, err)
   358→			return
   359→		}
   360→		cclog.Infof("NATS: transferred cached job to main table: old id %d -> new id %d (jobId=%d)", *job.ID, newID, job.JobID)
   361→		job.ID = &newID
   362→	}
   363→
   364→	if err := api.JobRepository.Stop(*job.ID, job.Duration, job.State, job.MonitoringStatus); err != nil {
   365→		cclog.Errorf("NATS job stop: jobId %d (id %d) on %s: marking job as '%s' failed: %v",
   366→			job.JobID, *job.ID, job.Cluster, job.State, err)
   367→		return
   368→	}
   369→
   370→	cclog.Infof("NATS: archiving job (dbid: %d): cluster=%s, jobId=%d, user=%s, startTime=%d, duration=%d, state=%s",
   371→		*job.ID, job.Cluster, job.JobID, job.User, job.StartTime, job.Duration, job.State)
   372→
   373→	if job.MonitoringStatus == schema.MonitoringStatusDisabled {
   374→		return
   375→	}
   376→
   377→	archiver.TriggerArchiving(job)
   378→}
   379→
   380→// processNodestateEvent extracts and processes node state data from the InfluxDB message.
   381→// Updates node states in the repository for all nodes in the payload.
   382→func (api *NatsAPI) processNodestateEvent(msg lp.CCMessage) {
   383→	v, ok := msg.GetEventValue()
   384→	if !ok {
   385→		cclog.Errorf("Nodestate event is missing event field with JSON payload")
   386→		return
   387→	}
   388→
   389→	var req UpdateNodeStatesRequest
   390→
   391→	dec := json.NewDecoder(strings.NewReader(v))
   392→	dec.DisallowUnknownFields()
   393→	if err := dec.Decode(&req); err != nil {
   394→		cclog.Errorf("NATS nodestate: parsing request failed: %v", err)
   395→		return
   396→	}
   397→
   398→	repo := repository.GetNodeRepository()
   399→	requestReceived := time.Now().Unix()
   400→
   401→	updates := make([]repository.NodeStateUpdate, 0, len(req.Nodes))
   402→	for _, node := range req.Nodes {
   403→		state := determineState(node.States)
   404→		nodeState := schema.NodeStateDB{
   405→			TimeStamp:       requestReceived,
   406→			NodeState:       state,
   407→			CpusAllocated:   node.CpusAllocated,
   408→			MemoryAllocated: node.MemoryAllocated,
   409→			GpusAllocated:   node.GpusAllocated,
   410→			HealthState:     schema.MonitoringStateFull,
   411→			JobsRunning:     node.JobsRunning,
   412→		}
   413→		updates = append(updates, repository.NodeStateUpdate{
   414→			Hostname:  node.Hostname,
   415→			Cluster:   req.Cluster,
   416→			NodeState: &nodeState,
   417→		})
   418→	}
   419→
   420→	if err := repo.BatchUpdateNodeStates(updates); err != nil {
   421→		cclog.Errorf("NATS nodestate: batch update for cluster %s failed: %v", req.Cluster, err)
   422→	}
   423→
   424→	cclog.Debugf("NATS nodestate: updated %d node states for cluster %s", len(req.Nodes), req.Cluster)
   425→}
   426→
   427→// handleNodeState processes node state update messages received via NATS using InfluxDB line protocol.
   428→// The message must be in line protocol format with measurement="nodestate" and include:
   429→//   - field "event" containing JSON payload (UpdateNodeStatesRequest)
   430→//
   431→// Example: nodestate event="{\"cluster\":\"testcluster\",\"nodes\":[...]}" 1234567890000000000
   432→func (api *NatsAPI) handleNodeState(subject string, data []byte) {
   433→	if len(data) == 0 {
   434→		cclog.Warnf("NATS %s: received empty message", subject)
   435→		return
   436→	}
   437→
   438→	d := influx.NewDecoderWithBytes(data)
   439→
   440→	for d.Next() {
   441→		m, err := receivers.DecodeInfluxMessage(d)
   442→		if err != nil {
   443→			cclog.Errorf("NATS %s: failed to decode InfluxDB line protocol message: %v", subject, err)
   444→			return
   445→		}
   446→
   447→		if !m.IsEvent() {
   448→			cclog.Warnf("NATS %s: received non-event message, skipping", subject)
   449→			continue
   450→		}
   451→
   452→		if m.Name() == "nodestate" {
   453→			api.processNodestateEvent(m)
   454→		} else {
   455→			cclog.Warnf("NATS %s: unexpected measurement name '%s', expected 'nodestate'", subject, m.Name())
   456→		}
   457→	}
   458→}
   459→


internal/api/node.go:23:type UpdateNodeStatesRequest struct {
internal/api/node.go:58:// @summary     Deliver updated Slurm node states
internal/api/node.go:63:// @param       request body UpdateNodeStatesRequest true "Request body containing nodes and their states"
internal/api/node.go:73:	req := UpdateNodeStatesRequest{}
internal/api/node.go:145:	if err := repo.BatchUpdateNodeStates(updates); err != nil {
internal/api/docs.go:914:                "summary": "Deliver updated Slurm node states",
internal/api/docs.go:922:                            "$ref": "#/definitions/api.UpdateNodeStatesRequest"
internal/api/docs.go:2093:        "api.UpdateNodeStatesRequest": {
internal/api/nats.go:59:// Node state updates use the "nodestate" measurement with cluster information:
internal/api/nats.go:63:// The JSON payload follows the UpdateNodeStatesRequest structure.
internal/api/nats.go:389:	var req UpdateNodeStatesRequest
internal/api/nats.go:420:	if err := repo.BatchUpdateNodeStates(updates); err != nil {
internal/api/nats.go:424:	cclog.Debugf("NATS nodestate: updated %d node states for cluster %s", len(req.Nodes), req.Cluster)
internal/api/nats.go:429://   - field "event" containing JSON payload (UpdateNodeStatesRequest)


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package api
     7→
     8→import (
     9→	"fmt"
    10→	"maps"
    11→	"net/http"
    12→	"strings"
    13→	"time"
    14→
    15→	"github.com/ClusterCockpit/cc-backend/internal/metricdispatch"
    16→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    17→	"github.com/ClusterCockpit/cc-backend/pkg/archive"
    18→	"github.com/ClusterCockpit/cc-backend/pkg/metricstore"
    19→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    20→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    21→)
    22→
    23→type UpdateNodeStatesRequest struct {
    24→	Nodes   []schema.NodePayload `json:"nodes"`
    25→	Cluster string               `json:"cluster" example:"fritz"`
    26→}
    27→
    28→// metricListToNames converts a map of metric configurations to a list of metric names
    29→func metricListToNames(metricList map[string]*schema.Metric) []string {
    30→	names := make([]string, 0, len(metricList))
    31→	for name := range metricList {
    32→		names = append(names, name)
    33→	}
    34→	return names
    35→}
    36→
    37→// this routine assumes that only one of them exists per node
    38→func determineState(states []string) schema.SchedulerState {
    39→	for _, state := range states {
    40→		switch strings.ToLower(state) {
    41→		case "allocated":
    42→			return schema.NodeStateAllocated
    43→		case "reserved":
    44→			return schema.NodeStateReserved
    45→		case "idle":
    46→			return schema.NodeStateIdle
    47→		case "down":
    48→			return schema.NodeStateDown
    49→		case "mixed":
    50→			return schema.NodeStateMixed
    51→		}
    52→	}
    53→
    54→	return schema.NodeStateUnknown
    55→}
    56→
    57→// updateNodeStates godoc
    58→// @summary     Deliver updated Slurm node states
    59→// @tags Nodestates
    60→// @description Returns a JSON-encoded list of users.
    61→// @description Required query-parameter defines if all users or only users with additional special roles are returned.
    62→// @produce     json
    63→// @param       request body UpdateNodeStatesRequest true "Request body containing nodes and their states"
    64→// @success     200     {object} api.DefaultAPIResponse "Success message"
    65→// @failure     400     {object} api.ErrorResponse      "Bad Request"
    66→// @failure     401     {object} api.ErrorResponse      "Unauthorized"
    67→// @failure     403     {object} api.ErrorResponse      "Forbidden"
    68→// @failure     500     {object} api.ErrorResponse      "Internal Server Error"
    69→// @security    ApiKeyAuth
    70→// @router      /api/nodestats/ [post]
    71→func (api *RestAPI) updateNodeStates(rw http.ResponseWriter, r *http.Request) {
    72→	// Parse request body
    73→	req := UpdateNodeStatesRequest{}
    74→	if err := decode(r.Body, &req); err != nil {
    75→		handleError(fmt.Errorf("parsing request body failed: %w", err),
    76→			http.StatusBadRequest, rw)
    77→		return
    78→	}
    79→	requestReceived := time.Now().Unix()
    80→	repo := repository.GetNodeRepository()
    81→
    82→	m := make(map[string][]string)
    83→	metricNames := make(map[string][]string)
    84→	healthResults := make(map[string]metricstore.HealthCheckResult)
    85→
    86→	startMs := time.Now()
    87→
    88→	// Step 1: Build nodeList and metricList per subcluster
    89→	for _, node := range req.Nodes {
    90→		if sc, err := archive.GetSubClusterByNode(req.Cluster, node.Hostname); err == nil {
    91→			m[sc] = append(m[sc], node.Hostname)
    92→		}
    93→	}
    94→
    95→	for sc := range m {
    96→		if sc != "" {
    97→			metricList := archive.GetMetricConfigSubCluster(req.Cluster, sc)
    98→			metricNames[sc] = metricListToNames(metricList)
    99→		}
   100→	}
   101→
   102→	// Step 2: Determine which metric store to query and perform health check
   103→	healthRepo, err := metricdispatch.GetHealthCheckRepo(req.Cluster)
   104→	if err != nil {
   105→		cclog.Warnf("updateNodeStates: no metric store for cluster %s, skipping health check: %v", req.Cluster, err)
   106→	} else {
   107→		for sc, nl := range m {
   108→			if sc != "" {
   109→				if results, err := healthRepo.HealthCheck(req.Cluster, nl, metricNames[sc]); err == nil {
   110→					maps.Copy(healthResults, results)
   111→				}
   112→			}
   113→		}
   114→	}
   115→
   116→	cclog.Debugf("Timer updateNodeStates, MemStore HealthCheck: %s", time.Since(startMs))
   117→	startDB := time.Now()
   118→
   119→	updates := make([]repository.NodeStateUpdate, 0, len(req.Nodes))
   120→	for _, node := range req.Nodes {
   121→		state := determineState(node.States)
   122→		healthState := schema.MonitoringStateFailed
   123→		var healthMetrics string
   124→		if result, ok := healthResults[node.Hostname]; ok {
   125→			healthState = result.State
   126→			healthMetrics = result.HealthMetrics
   127→		}
   128→		nodeState := schema.NodeStateDB{
   129→			TimeStamp:       requestReceived,
   130→			NodeState:       state,
   131→			CpusAllocated:   node.CpusAllocated,
   132→			MemoryAllocated: node.MemoryAllocated,
   133→			GpusAllocated:   node.GpusAllocated,
   134→			HealthState:     healthState,
   135→			HealthMetrics:   healthMetrics,
   136→			JobsRunning:     node.JobsRunning,
   137→		}
   138→		updates = append(updates, repository.NodeStateUpdate{
   139→			Hostname:  node.Hostname,
   140→			Cluster:   req.Cluster,
   141→			NodeState: &nodeState,
   142→		})
   143→	}
   144→
   145→	if err := repo.BatchUpdateNodeStates(updates); err != nil {
   146→		cclog.Errorf("updateNodeStates: batch update for cluster %s failed: %v", req.Cluster, err)
   147→	}
   148→
   149→	cclog.Debugf("Timer updateNodeStates, SQLite Inserts: %s", time.Since(startDB))
   150→}
   151→


No files found


Found 6 files
internal/api/nats.go
internal/config/config.go
internal/api/node.go
cmd/cc-backend/server.go
internal/api/rest.go
internal/api/docs.go


84:	r.Post("/nodestate/", api.updateNodeStates)
85:	r.Put("/nodestate/", api.updateNodeStates)
134:	r.Post("/healthcheck", api.updateNodeStates)
139:	r.Post("/healthcheck/", api.updateNodeStates)


grep -n "BatchUpdateNodeStates\|NodeStateUpdate" /Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/*.go


/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:247:// NodeStateUpdate holds the data needed to update one node's state in a batch operation.
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:248:type NodeStateUpdate struct {
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:254:// BatchUpdateNodeStates inserts node state rows for multiple nodes in a single transaction.
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:257:func (r *NodeRepository) BatchUpdateNodeStates(updates []NodeStateUpdate) error {
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:260:		return fmt.Errorf("BatchUpdateNodeStates: begin transaction: %w", err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:266:		return fmt.Errorf("BatchUpdateNodeStates: prepare lookup: %w", err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:272:		return fmt.Errorf("BatchUpdateNodeStates: prepare node insert: %w", err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:278:		return fmt.Errorf("BatchUpdateNodeStates: prepare state insert: %w", err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:288:					cclog.Errorf("BatchUpdateNodeStates: subcluster lookup for '%s' in '%s': %v", u.Hostname, u.Cluster, scErr)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:296:					cclog.Errorf("BatchUpdateNodeStates: insert node '%s': %v", u.Hostname, insertErr)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:301:				cclog.Errorf("BatchUpdateNodeStates: lookup node '%s': %v", u.Hostname, err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:308:			cclog.Errorf("BatchUpdateNodeStates: insert state for '%s': %v", u.Hostname, err)
/Users/jan/prg/ClusterCockpit/cc-backend/internal/repository/node.go:313:		return fmt.Errorf("BatchUpdateNodeStates: commit: %w", err)


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package repository
     7→
     8→import (
     9→	"context"
    10→	"database/sql"
    11→	"encoding/json"
    12→	"fmt"
    13→	"slices"
    14→	"sort"
    15→	"strings"
    16→	"sync"
    17→	"time"
    18→
    19→	"github.com/ClusterCockpit/cc-backend/internal/graph/model"
    20→	"github.com/ClusterCockpit/cc-backend/pkg/archive"
    21→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    22→	"github.com/ClusterCockpit/cc-lib/v2/lrucache"
    23→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    24→	sq "github.com/Masterminds/squirrel"
    25→	"github.com/jmoiron/sqlx"
    26→)
    27→
    28→var (
    29→	nodeRepoOnce     sync.Once
    30→	nodeRepoInstance *NodeRepository
    31→)
    32→
    33→type NodeRepository struct {
    34→	DB        *sqlx.DB
    35→	stmtCache *sq.StmtCache
    36→	cache     *lrucache.Cache
    37→	driver    string
    38→}
    39→
    40→func GetNodeRepository() *NodeRepository {
    41→	nodeRepoOnce.Do(func() {
    42→		db := GetConnection()
    43→
    44→		nodeRepoInstance = &NodeRepository{
    45→			DB:     db.DB,
    46→			driver: db.Driver,
    47→
    48→			stmtCache: sq.NewStmtCache(db.DB),
    49→			cache:     lrucache.New(repoConfig.CacheSize),
    50→		}
    51→	})
    52→	return nodeRepoInstance
    53→}
    54→
    55→// latestStateCondition returns a squirrel expression that restricts node_state
    56→// rows to the latest per node_id using a correlated subquery.
    57→// Requires the query to join node and node_state tables.
    58→func latestStateCondition() sq.Sqlizer {
    59→	return sq.Expr(
    60→		"node_state.id = (SELECT ns2.id FROM node_state ns2 WHERE ns2.node_id = node.id ORDER BY ns2.time_stamp DESC LIMIT 1)",
    61→	)
    62→}
    63→
    64→// applyNodeFilters applies common NodeFilter conditions to a query that joins
    65→// the node and node_state tables with latestStateCondition.
    66→func applyNodeFilters(query sq.SelectBuilder, filters []*model.NodeFilter) sq.SelectBuilder {
    67→	for _, f := range filters {
    68→		if f.Cluster != nil {
    69→			query = buildStringCondition("node.cluster", f.Cluster, query)
    70→		}
    71→		if f.SubCluster != nil {
    72→			query = buildStringCondition("node.subcluster", f.SubCluster, query)
    73→		}
    74→		if f.Hostname != nil {
    75→			query = buildStringCondition("node.hostname", f.Hostname, query)
    76→		}
    77→		if f.SchedulerState != nil {
    78→			query = query.Where("node_state.node_state = ?", f.SchedulerState)
    79→		}
    80→		if f.HealthState != nil {
    81→			query = query.Where("node_state.health_state = ?", f.HealthState)
    82→		}
    83→	}
    84→	return query
    85→}
    86→
    87→func (r *NodeRepository) FetchMetadata(hostname string, cluster string) (map[string]string, error) {
    88→	start := time.Now()
    89→
    90→	RawMetaData := make([]byte, 0)
    91→
    92→	if err := sq.Select("node.meta_data").From("node").
    93→		Where("node.hostname = ?", hostname).
    94→		Where("node.cluster = ?", cluster).
    95→		RunWith(r.stmtCache).QueryRow().Scan(&RawMetaData); err != nil {
    96→		cclog.Warn("Error while scanning for node metadata")
    97→		return nil, err
    98→	}
    99→
   100→	if len(RawMetaData) == 0 {
   101→		return nil, nil
   102→	}
   103→
   104→	MetaData := make(map[string]string)
   105→
   106→	if err := json.Unmarshal(RawMetaData, &MetaData); err != nil {
   107→		cclog.Warn("Error while unmarshaling raw metadata json")
   108→		return nil, err
   109→	}
   110→
   111→	cclog.Debugf("Timer FetchMetadata %s", time.Since(start))
   112→	return MetaData, nil
   113→}
   114→
   115→func (r *NodeRepository) GetNode(hostname string, cluster string, withMeta bool) (*schema.Node, error) {
   116→	node := &schema.Node{}
   117→	if err := sq.Select("node.hostname", "node.cluster", "node.subcluster",
   118→		"node_state.node_state", "node_state.health_state").
   119→		From("node").
   120→		Join("node_state ON node_state.node_id = node.id").
   121→		Where(latestStateCondition()).
   122→		Where("node.hostname = ?", hostname).
   123→		Where("node.cluster = ?", cluster).
   124→		RunWith(r.DB).
   125→		QueryRow().Scan(&node.Hostname, &node.Cluster, &node.SubCluster, &node.NodeState, &node.HealthState); err != nil {
   126→		cclog.Warnf("Error while querying node '%s' from database: %v", hostname, err)
   127→		return nil, err
   128→	}
   129→
   130→	if withMeta {
   131→		var err error
   132→		var meta map[string]string
   133→		if meta, err = r.FetchMetadata(hostname, cluster); err != nil {
   134→			cclog.Warnf("Error while fetching metadata for node '%s'", hostname)
   135→			return nil, err
   136→		}
   137→		node.MetaData = meta
   138→	}
   139→
   140→	return node, nil
   141→}
   142→
   143→func (r *NodeRepository) GetNodeByID(id int64, withMeta bool) (*schema.Node, error) {
   144→	node := &schema.Node{}
   145→	if err := sq.Select("node.hostname", "node.cluster", "node.subcluster",
   146→		"node_state.node_state", "node_state.health_state").
   147→		From("node").
   148→		Join("node_state ON node_state.node_id = node.id").
   149→		Where(latestStateCondition()).
   150→		Where("node.id = ?", id).
   151→		RunWith(r.DB).
   152→		QueryRow().Scan(&node.Hostname, &node.Cluster, &node.SubCluster, &node.NodeState, &node.HealthState); err != nil {
   153→		cclog.Warnf("Error while querying node ID '%d' from database: %v", id, err)
   154→		return nil, err
   155→	}
   156→
   157→	if withMeta {
   158→		meta, metaErr := r.FetchMetadata(node.Hostname, node.Cluster)
   159→		if metaErr != nil {
   160→			cclog.Warnf("Error while fetching metadata for node ID '%d': %v", id, metaErr)
   161→			return nil, metaErr
   162→		}
   163→		node.MetaData = meta
   164→	}
   165→
   166→	return node, nil
   167→}
   168→
   169→// const NamedNodeInsert string = `
   170→// INSERT INTO node (time_stamp, hostname, cluster, subcluster, node_state, health_state,
   171→//
   172→//	cpus_allocated, cpus_total, memory_allocated, memory_total, gpus_allocated, gpus_total)
   173→//	VALUES (:time_stamp, :hostname, :cluster, :subcluster, :node_state, :health_state,
   174→//	:cpus_allocated, :cpus_total, :memory_allocated, :memory_total, :gpus_allocated, :gpus_total);`
   175→
   176→const NamedNodeInsert string = `
   177→INSERT INTO node (hostname, cluster, subcluster)
   178→	VALUES (:hostname, :cluster, :subcluster);`
   179→
   180→// AddNode adds a Node to the node table. This can be triggered by a node collector registration or
   181→// from a nodestate update from the job scheduler.
   182→func (r *NodeRepository) AddNode(node *schema.NodeDB) (int64, error) {
   183→	var err error
   184→
   185→	res, err := r.DB.NamedExec(NamedNodeInsert, node)
   186→	if err != nil {
   187→		cclog.Errorf("Error while adding node '%v' to database", node.Hostname)
   188→		return 0, err
   189→	}
   190→	node.ID, err = res.LastInsertId()
   191→	if err != nil {
   192→		cclog.Errorf("Error while getting last insert id for node '%v' from database", node.Hostname)
   193→		return 0, err
   194→	}
   195→
   196→	return node.ID, nil
   197→}
   198→
   199→const NamedNodeStateInsert string = `
   200→INSERT INTO node_state (time_stamp, node_state, health_state, health_metrics,
   201→	cpus_allocated, memory_allocated, gpus_allocated, jobs_running, node_id)
   202→	VALUES (:time_stamp, :node_state, :health_state, :health_metrics,
   203→	:cpus_allocated, :memory_allocated, :gpus_allocated, :jobs_running, :node_id);`
   204→
   205→// TODO: Add real Monitoring Health State
   206→
   207→// UpdateNodeState is called from the Node REST API to add a row in the node state table
   208→func (r *NodeRepository) UpdateNodeState(hostname string, cluster string, nodeState *schema.NodeStateDB) error {
   209→	var id int64
   210→
   211→	if err := sq.Select("id").From("node").
   212→		Where("node.hostname = ?", hostname).Where("node.cluster = ?", cluster).RunWith(r.DB).
   213→		QueryRow().Scan(&id); err != nil {
   214→		if err == sql.ErrNoRows {
   215→			subcluster, err := archive.GetSubClusterByNode(cluster, hostname)
   216→			if err != nil {
   217→				cclog.Errorf("Error while getting subcluster for node '%s' in cluster '%s': %v", hostname, cluster, err)
   218→				return err
   219→			}
   220→			node := schema.NodeDB{
   221→				Hostname: hostname, Cluster: cluster, SubCluster: subcluster,
   222→			}
   223→			id, err = r.AddNode(&node)
   224→			if err != nil {
   225→				cclog.Errorf("Error while adding node '%s' to database: %v", hostname, err)
   226→				return err
   227→			}
   228→
   229→			cclog.Debugf("Added node '%s' to database", hostname)
   230→		} else {
   231→			cclog.Warnf("Error while querying node '%v' from database", id)
   232→			return err
   233→		}
   234→	}
   235→
   236→	nodeState.NodeID = id
   237→
   238→	_, err := r.DB.NamedExec(NamedNodeStateInsert, nodeState)
   239→	if err != nil {
   240→		cclog.Errorf("Error while adding node state for '%v' to database", hostname)
   241→		return err
   242→	}
   243→	cclog.Debugf("Updated node state for '%s' in database", hostname)
   244→	return nil
   245→}
   246→
   247→// NodeStateUpdate holds the data needed to update one node's state in a batch operation.
   248→type NodeStateUpdate struct {
   249→	Hostname  string
   250→	Cluster   string
   251→	NodeState *schema.NodeStateDB
   252→}
   253→
   254→// BatchUpdateNodeStates inserts node state rows for multiple nodes in a single transaction.
   255→// For each node, it looks up (or creates) the node row, then inserts the state row.
   256→// This reduces lock acquisitions from 2*N to 1 for N nodes.
   257→func (r *NodeRepository) BatchUpdateNodeStates(updates []NodeStateUpdate) error {
   258→	tx, err := r.DB.Beginx()
   259→	if err != nil {
   260→		return fmt.Errorf("BatchUpdateNodeStates: begin transaction: %w", err)
   261→	}
   262→	defer tx.Rollback()
   263→
   264→	stmtLookup, err := tx.Preparex("SELECT id FROM node WHERE hostname = ? AND cluster = ?")
   265→	if err != nil {
   266→		return fmt.Errorf("BatchUpdateNodeStates: prepare lookup: %w", err)
   267→	}
   268→	defer stmtLookup.Close()
   269→
   270→	stmtInsertNode, err := tx.PrepareNamed(NamedNodeInsert)
   271→	if err != nil {
   272→		return fmt.Errorf("BatchUpdateNodeStates: prepare node insert: %w", err)
   273→	}
   274→	defer stmtInsertNode.Close()
   275→
   276→	stmtInsertState, err := tx.PrepareNamed(NamedNodeStateInsert)
   277→	if err != nil {
   278→		return fmt.Errorf("BatchUpdateNodeStates: prepare state insert: %w", err)
   279→	}
   280→	defer stmtInsertState.Close()
   281→
   282→	for _, u := range updates {
   283→		var id int64
   284→		if err := stmtLookup.QueryRow(u.Hostname, u.Cluster).Scan(&id); err != nil {
   285→			if err == sql.ErrNoRows {
   286→				subcluster, scErr := archive.GetSubClusterByNode(u.Cluster, u.Hostname)
   287→				if scErr != nil {
   288→					cclog.Errorf("BatchUpdateNodeStates: subcluster lookup for '%s' in '%s': %v", u.Hostname, u.Cluster, scErr)
   289→					continue
   290→				}
   291→				node := schema.NodeDB{
   292→					Hostname: u.Hostname, Cluster: u.Cluster, SubCluster: subcluster,
   293→				}
   294→				res, insertErr := stmtInsertNode.Exec(&node)
   295→				if insertErr != nil {
   296→					cclog.Errorf("BatchUpdateNodeStates: insert node '%s': %v", u.Hostname, insertErr)
   297→					continue
   298→				}
   299→				id, _ = res.LastInsertId()
   300→			} else {
   301→				cclog.Errorf("BatchUpdateNodeStates: lookup node '%s': %v", u.Hostname, err)
   302→				continue
   303→			}
   304→		}
   305→
   306→		u.NodeState.NodeID = id
   307→		if _, err := stmtInsertState.Exec(u.NodeState); err != nil {
   308→			cclog.Errorf("BatchUpdateNodeStates: insert state for '%s': %v", u.Hostname, err)
   309→		}
   310→	}
   311→
   312→	if err := tx.Commit(); err != nil {
   313→		return fmt.Errorf("BatchUpdateNodeStates: commit: %w", err)
   314→	}
   315→	return nil
   316→}
   317→
   318→// func (r *NodeRepository) UpdateHealthState(hostname string, healthState *schema.MonitoringState) error {
   319→// 	if _, err := sq.Update("node").Set("health_state", healthState).Where("node.id = ?", id).RunWith(r.DB).Exec(); err != nil {
   320→// 		cclog.Errorf("error while updating node '%d'", id)
   321→// 		return err
   322→// 	}
   323→//
   324→// 	return nil
   325→// }
   326→
   327→// NodeStateWithNode combines a node state row with denormalized node info.
   328→type NodeStateWithNode struct {
   329→	ID              int64  `db:"id"`
   330→	TimeStamp       int64  `db:"time_stamp"`
   331→	NodeState       string `db:"node_state"`
   332→	HealthState     string `db:"health_state"`
   333→	HealthMetrics   string `db:"health_metrics"`
   334→	CpusAllocated   int    `db:"cpus_allocated"`
   335→	MemoryAllocated int64  `db:"memory_allocated"`
   336→	GpusAllocated   int    `db:"gpus_allocated"`
   337→	JobsRunning     int    `db:"jobs_running"`
   338→	Hostname        string `db:"hostname"`
   339→	Cluster         string `db:"cluster"`
   340→	SubCluster      string `db:"subcluster"`
   341→}
   342→
   343→// FindNodeStatesBefore returns all node_state rows with time_stamp < cutoff,
   344→// joined with node info for denormalized archiving.
   345→func (r *NodeRepository) FindNodeStatesBefore(cutoff int64) ([]NodeStateWithNode, error) {
   346→	rows, err := sq.Select(
   347→		"node_state.id", "node_state.time_stamp", "node_state.node_state",
   348→		"node_state.health_state", "node_state.health_metrics",
   349→		"node_state.cpus_allocated", "node_state.memory_allocated",
   350→		"node_state.gpus_allocated", "node_state.jobs_running",
   351→		"node.hostname", "node.cluster", "node.subcluster",
   352→	).
   353→		From("node_state").
   354→		Join("node ON node_state.node_id = node.id").
   355→		Where(sq.Lt{"node_state.time_stamp": cutoff}).
   356→		Where("node_state.id NOT IN (SELECT ns2.id FROM node_state ns2 WHERE ns2.time_stamp = (SELECT MAX(ns3.time_stamp) FROM node_state ns3 WHERE ns3.node_id = ns2.node_id))").
   357→		OrderBy("node.cluster ASC", "node.subcluster ASC", "node.hostname ASC", "node_state.time_stamp ASC").
   358→		RunWith(r.DB).Query()
   359→	if err != nil {
   360→		return nil, err
   361→	}
   362→	defer rows.Close()
   363→
   364→	var result []NodeStateWithNode
   365→	for rows.Next() {
   366→		var ns NodeStateWithNode
   367→		var healthMetrics sql.NullString
   368→		if err := rows.Scan(&ns.ID, &ns.TimeStamp, &ns.NodeState,
   369→			&ns.HealthState, &healthMetrics,
   370→			&ns.CpusAllocated, &ns.MemoryAllocated,
   371→			&ns.GpusAllocated, &ns.JobsRunning,
   372→			&ns.Hostname, &ns.Cluster, &ns.SubCluster); err != nil {
   373→			return nil, err
   374→		}
   375→		ns.HealthMetrics = healthMetrics.String
   376→		result = append(result, ns)
   377→	}
   378→	return result, nil
   379→}
   380→
   381→// DeleteNodeStatesBefore removes node_state rows with time_stamp < cutoff,
   382→// but always preserves the row with the latest timestamp per node_id.
   383→func (r *NodeRepository) DeleteNodeStatesBefore(cutoff int64) (int64, error) {
   384→	res, err := r.DB.Exec(
   385→		`DELETE FROM node_state WHERE time_stamp < ?
   386→		 AND id NOT IN (
   387→		   SELECT id FROM node_state ns2
   388→		   WHERE ns2.time_stamp = (SELECT MAX(ns3.time_stamp) FROM node_state ns3 WHERE ns3.node_id = ns2.node_id)
   389→		 )`,
   390→		cutoff,
   391→	)
   392→	if err != nil {
   393→		return 0, err
   394→	}
   395→	return res.RowsAffected()
   396→}
   397→
   398→func (r *NodeRepository) DeleteNode(id int64) error {
   399→	_, err := r.DB.Exec(`DELETE FROM node WHERE node.id = ?`, id)
   400→	if err != nil {
   401→		cclog.Errorf("Error while deleting node '%d' from DB", id)
   402→		return err
   403→	}
   404→	cclog.Infof("deleted node '%d' from DB", id)
   405→	return nil
   406→}
   407→
   408→// QueryNodes returns a list of nodes based on a node filter. It always operates
   409→// on the last state (largest timestamp).
   410→func (r *NodeRepository) QueryNodes(
   411→	ctx context.Context,
   412→	filters []*model.NodeFilter,
   413→	page *model.PageRequest,
   414→	order *model.OrderByInput, // Currently unused!
   415→) ([]*schema.Node, error) {
   416→	query, qerr := AccessCheck(ctx,
   417→		sq.Select("node.hostname", "node.cluster", "node.subcluster",
   418→			"node_state.node_state", "node_state.health_state").
   419→			From("node").
   420→			Join("node_state ON node_state.node_id = node.id").
   421→			Where(latestStateCondition()))
   422→	if qerr != nil {
   423→		return nil, qerr
   424→	}
   425→
   426→	query = applyNodeFilters(query, filters)
   427→	query = query.OrderBy("node.hostname ASC")
   428→
   429→	if page != nil && page.ItemsPerPage != -1 {
   430→		limit := uint64(page.ItemsPerPage)
   431→		query = query.Offset((uint64(page.Page) - 1) * limit).Limit(limit)
   432→	}
   433→
   434→	rows, err := query.RunWith(r.stmtCache).Query()
   435→	if err != nil {
   436→		queryString, queryVars, _ := query.ToSql()
   437→		cclog.Errorf("Error while running query '%s' %v: %v", queryString, queryVars, err)
   438→		return nil, err
   439→	}
   440→	defer rows.Close()
   441→
   442→	nodes := make([]*schema.Node, 0)
   443→	for rows.Next() {
   444→		node := schema.Node{}
   445→		if err := rows.Scan(&node.Hostname, &node.Cluster, &node.SubCluster,
   446→			&node.NodeState, &node.HealthState); err != nil {
   447→			cclog.Warn("Error while scanning rows (QueryNodes)")
   448→			return nil, err
   449→		}
   450→		nodes = append(nodes, &node)
   451→	}
   452→
   453→	if err := rows.Err(); err != nil {
   454→		return nil, err
   455→	}
   456→
   457→	return nodes, nil
   458→}
   459→
   460→// QueryNodesWithMeta returns a list of nodes based on a node filter. It always operates
   461→// on the last state (largest timestamp). It includes both (!) optional JSON column data
   462→func (r *NodeRepository) QueryNodesWithMeta(
   463→	ctx context.Context,
   464→	filters []*model.NodeFilter,
   465→	page *model.PageRequest,
   466→	order *model.OrderByInput, // Currently unused!
   467→) ([]*schema.Node, error) {
   468→	query, qerr := AccessCheck(ctx,
   469→		sq.Select("node.hostname", "node.cluster", "node.subcluster",
   470→			"node_state.node_state", "node_state.health_state",
   471→			"node.meta_data", "node_state.health_metrics").
   472→			From("node").
   473→			Join("node_state ON node_state.node_id = node.id").
   474→			Where(latestStateCondition()))
   475→	if qerr != nil {
   476→		return nil, qerr
   477→	}
   478→
   479→	query = applyNodeFilters(query, filters)
   480→	query = query.OrderBy("node.hostname ASC")
   481→
   482→	if page != nil && page.ItemsPerPage != -1 {
   483→		limit := uint64(page.ItemsPerPage)
   484→		query = query.Offset((uint64(page.Page) - 1) * limit).Limit(limit)
   485→	}
   486→
   487→	rows, err := query.RunWith(r.stmtCache).Query()
   488→	if err != nil {
   489→		queryString, queryVars, _ := query.ToSql()
   490→		cclog.Errorf("Error while running query '%s' %v: %v", queryString, queryVars, err)
   491→		return nil, err
   492→	}
   493→	defer rows.Close()
   494→
   495→	nodes := make([]*schema.Node, 0)
   496→	for rows.Next() {
   497→		node := schema.Node{}
   498→		RawMetaData := make([]byte, 0)
   499→		RawMetricHealth := make([]byte, 0)
   500→
   501→		if err := rows.Scan(&node.Hostname, &node.Cluster, &node.SubCluster,
   502→			&node.NodeState, &node.HealthState, &RawMetaData, &RawMetricHealth); err != nil {
   503→			cclog.Warn("Error while scanning rows (QueryNodes)")
   504→			return nil, err
   505→		}
   506→
   507→		if len(RawMetaData) == 0 {
   508→			node.MetaData = nil
   509→		} else {
   510→			metaData := make(map[string]string)
   511→			if err := json.Unmarshal(RawMetaData, &metaData); err != nil {
   512→				cclog.Warn("Error while unmarshaling raw metadata json")
   513→				return nil, err
   514→			}
   515→			node.MetaData = metaData
   516→		}
   517→
   518→		if len(RawMetricHealth) == 0 {
   519→			node.HealthData = nil
   520→		} else {
   521→			healthData := make(map[string][]string)
   522→			if err := json.Unmarshal(RawMetricHealth, &healthData); err != nil {
   523→				cclog.Warn("Error while unmarshaling raw healthdata json")
   524→				return nil, err
   525→			}
   526→			node.HealthData = healthData
   527→		}
   528→
   529→		nodes = append(nodes, &node)
   530→	}
   531→
   532→	if err := rows.Err(); err != nil {
   533→		return nil, err
   534→	}
   535→
   536→	return nodes, nil
   537→}
   538→
   539→// CountNodes returns the total matched nodes based on a node filter. It always operates
   540→// on the last state (largest timestamp) per node.
   541→func (r *NodeRepository) CountNodes(
   542→	ctx context.Context,
   543→	filters []*model.NodeFilter,
   544→) (int, error) {
   545→	query, qerr := AccessCheck(ctx,
   546→		sq.Select("COUNT(*)").
   547→			From("node").
   548→			Join("node_state ON node_state.node_id = node.id").
   549→			Where(latestStateCondition()))
   550→	if qerr != nil {
   551→		return 0, qerr
   552→	}
   553→
   554→	query = applyNodeFilters(query, filters)
   555→
   556→	var count int
   557→	if err := query.RunWith(r.stmtCache).QueryRow().Scan(&count); err != nil {
   558→		queryString, queryVars, _ := query.ToSql()
   559→		cclog.Errorf("Error while running query '%s' %v: %v", queryString, queryVars, err)
   560→		return 0, err
   561→	}
   562→
   563→	return count, nil
   564→}
   565→
   566→func (r *NodeRepository) ListNodes(cluster string) ([]*schema.Node, error) {
   567→	q := sq.Select("node.hostname", "node.cluster", "node.subcluster",
   568→		"node_state.node_state", "node_state.health_state").
   569→		From("node").
   570→		Join("node_state ON node_state.node_id = node.id").
   571→		Where(latestStateCondition()).
   572→		Where("node.cluster = ?", cluster).
   573→		OrderBy("node.hostname ASC")
   574→
   575→	rows, err := q.RunWith(r.DB).Query()
   576→	if err != nil {
   577→		cclog.Warn("Error while querying node list")
   578→		return nil, err
   579→	}
   580→	nodeList := make([]*schema.Node, 0, 100)
   581→	defer rows.Close()
   582→	for rows.Next() {
   583→		node := &schema.Node{}
   584→		if err := rows.Scan(&node.Hostname, &node.Cluster,
   585→			&node.SubCluster, &node.NodeState, &node.HealthState); err != nil {
   586→			cclog.Warn("Error while scanning node list (ListNodes)")
   587→			return nil, err
   588→		}
   589→
   590→		nodeList = append(nodeList, node)
   591→	}
   592→
   593→	return nodeList, nil
   594→}
   595→
   596→func (r *NodeRepository) MapNodes(cluster string) (map[string]string, error) {
   597→	q := sq.Select("node.hostname", "node_state.node_state").
   598→		From("node").
   599→		Join("node_state ON node_state.node_id = node.id").
   600→		Where(latestStateCondition()).
   601→		Where("node.cluster = ?", cluster).
   602→		OrderBy("node.hostname ASC")
   603→
   604→	rows, err := q.RunWith(r.DB).Query()
   605→	if err != nil {
   606→		cclog.Warn("Error while querying node list")
   607→		return nil, err
   608→	}
   609→
   610→	stateMap := make(map[string]string)
   611→	defer rows.Close()
   612→	for rows.Next() {
   613→		var hostname, nodestate string
   614→		if err := rows.Scan(&hostname, &nodestate); err != nil {
   615→			cclog.Warn("Error while scanning node list (MapNodes)")
   616→			return nil, err
   617→		}
   618→
   619→		stateMap[hostname] = nodestate
   620→	}
   621→
   622→	return stateMap, nil
   623→}
   624→
   625→func (r *NodeRepository) CountStates(ctx context.Context, filters []*model.NodeFilter, column string) ([]*model.NodeStates, error) {
   626→	query, qerr := AccessCheck(ctx,
   627→		sq.Select(column, "COUNT(*) as count").
   628→			From("node").
   629→			Join("node_state ON node_state.node_id = node.id").
   630→			Where(latestStateCondition()).
   631→			GroupBy(column))
   632→	if qerr != nil {
   633→		return nil, qerr
   634→	}
   635→
   636→	query = applyNodeFilters(query, filters)
   637→
   638→	rows, err := query.RunWith(r.stmtCache).Query()
   639→	if err != nil {
   640→		queryString, queryVars, _ := query.ToSql()
   641→		cclog.Errorf("Error while running query '%s' %v: %v", queryString, queryVars, err)
   642→		return nil, err
   643→	}
   644→	defer rows.Close()
   645→
   646→	nodes := make([]*model.NodeStates, 0)
   647→	for rows.Next() {
   648→		var state string
   649→		var count int
   650→		if err := rows.Scan(&state, &count); err != nil {
   651→			cclog.Warn("Error while scanning rows (CountStates)")
   652→			return nil, err
   653→		}
   654→		nodes = append(nodes, &model.NodeStates{State: state, Count: count})
   655→	}
   656→
   657→	if err := rows.Err(); err != nil {
   658→		return nil, err
   659→	}
   660→
   661→	return nodes, nil
   662→}
   663→
   664→func (r *NodeRepository) CountStatesTimed(ctx context.Context, filters []*model.NodeFilter, column string) ([]*model.NodeStatesTimed, error) {
   665→	query, qerr := AccessCheck(ctx, sq.Select(column, "time_stamp", "count(*) as count").From("node")) // "cluster"?
   666→	if qerr != nil {
   667→		return nil, qerr
   668→	}
   669→
   670→	query = query.Join("node_state ON node_state.node_id = node.id")
   671→
   672→	for _, f := range filters {
   673→		// Required
   674→		if f.TimeStart != nil {
   675→			query = query.Where("time_stamp > ?", f.TimeStart)
   676→		}
   677→		// Optional
   678→		if f.Hostname != nil {
   679→			query = buildStringCondition("hostname", f.Hostname, query)
   680→		}
   681→		if f.Cluster != nil {
   682→			query = buildStringCondition("cluster", f.Cluster, query)
   683→		}
   684→		if f.SubCluster != nil {
   685→			query = buildStringCondition("subcluster", f.SubCluster, query)
   686→		}
   687→		if f.SchedulerState != nil {
   688→			query = query.Where("node_state = ?", f.SchedulerState)
   689→		}
   690→		if f.HealthState != nil {
   691→			query = query.Where("health_state = ?", f.HealthState)
   692→		}
   693→	}
   694→
   695→	// Add Group and Order
   696→	query = query.GroupBy(column + ", time_stamp").OrderBy("time_stamp ASC")
   697→
   698→	rows, err := query.RunWith(r.stmtCache).Query()
   699→	if err != nil {
   700→		queryString, queryVars, _ := query.ToSql()
   701→		cclog.Errorf("Error while running query '%s' %v: %v", queryString, queryVars, err)
   702→		return nil, err
   703→	}
   704→	defer rows.Close()
   705→
   706→	rawData := make(map[string][][]int)
   707→	for rows.Next() {
   708→		var state string
   709→		var timestamp, count int
   710→
   711→		if err := rows.Scan(&state, &timestamp, &count); err != nil {
   712→			cclog.Warnf("Error while scanning rows (CountStatesTimed) at time '%d'", timestamp)
   713→			return nil, err
   714→		}
   715→
   716→		if rawData[state] == nil {
   717→			rawData[state] = [][]int{make([]int, 0), make([]int, 0)}
   718→		}
   719→
   720→		rawData[state][0] = append(rawData[state][0], timestamp)
   721→		rawData[state][1] = append(rawData[state][1], count)
   722→	}
   723→
   724→	if err := rows.Err(); err != nil {
   725→		return nil, err
   726→	}
   727→
   728→	timedStates := make([]*model.NodeStatesTimed, 0)
   729→	for state, data := range rawData {
   730→		entry := model.NodeStatesTimed{State: state, Times: data[0], Counts: data[1]}
   731→		timedStates = append(timedStates, &entry)
   732→	}
   733→
   734→	return timedStates, nil
   735→}
   736→
   737→func (r *NodeRepository) GetNodesForList(
   738→	ctx context.Context,
   739→	cluster string,
   740→	subCluster string,
   741→	stateFilter string,
   742→	nodeFilter string,
   743→	page *model.PageRequest,
   744→) ([]string, map[string]string, int, bool, error) {
   745→	// Init Return Vars
   746→	nodes := make([]string, 0)
   747→	stateMap := make(map[string]string)
   748→	countNodes := 0
   749→	hasNextPage := false
   750→
   751→	// Build Filters
   752→	queryFilters := make([]*model.NodeFilter, 0)
   753→	if cluster != "" {
   754→		queryFilters = append(queryFilters, &model.NodeFilter{Cluster: &model.StringInput{Eq: &cluster}})
   755→	}
   756→	if subCluster != "" {
   757→		queryFilters = append(queryFilters, &model.NodeFilter{SubCluster: &model.StringInput{Eq: &subCluster}})
   758→	}
   759→	if nodeFilter != "" && stateFilter != "notindb" {
   760→		queryFilters = append(queryFilters, &model.NodeFilter{Hostname: &model.StringInput{Contains: &nodeFilter}})
   761→	}
   762→	if stateFilter != "all" && stateFilter != "notindb" {
   763→		queryState := schema.SchedulerState(stateFilter)
   764→		queryFilters = append(queryFilters, &model.NodeFilter{SchedulerState: &queryState})
   765→	}
   766→	// if healthFilter != "all" {
   767→	// 	filters = append(filters, &model.NodeFilter{HealthState: &healthFilter})
   768→	// }
   769→
   770→	// Special Case: Disable Paging for missing nodes filter, save IPP for later
   771→	var backupItems int
   772→	if stateFilter == "notindb" {
   773→		backupItems = page.ItemsPerPage
   774→		page.ItemsPerPage = -1
   775→	}
   776→
   777→	// Query Nodes From DB
   778→	rawNodes, serr := r.QueryNodes(ctx, queryFilters, page, nil) // Order not Used
   779→	if serr != nil {
   780→		cclog.Warn("error while loading node database data (Resolver.NodeMetricsList)")
   781→		return nil, nil, 0, false, serr
   782→	}
   783→
   784→	// Intermediate Node Result Info
   785→	for _, node := range rawNodes {
   786→		if node == nil {
   787→			continue
   788→		}
   789→		nodes = append(nodes, node.Hostname)
   790→		stateMap[node.Hostname] = string(node.NodeState)
   791→	}
   792→
   793→	// Special Case: Find Nodes not in DB node table but in metricStore only
   794→	if stateFilter == "notindb" {
   795→		// Reapply Original Paging
   796→		page.ItemsPerPage = backupItems
   797→		// Get Nodes From Topology
   798→		var topoNodes []string
   799→		if subCluster != "" {
   800→			scNodes := archive.NodeLists[cluster][subCluster]
   801→			topoNodes = scNodes.PrintList()
   802→		} else {
   803→			subClusterNodeLists := archive.NodeLists[cluster]
   804→			for _, nodeList := range subClusterNodeLists {
   805→				topoNodes = append(topoNodes, nodeList.PrintList()...)
   806→			}
   807→		}
   808→		// Compare to all nodes from cluster/subcluster in DB
   809→		var missingNodes []string
   810→		for _, scanNode := range topoNodes {
   811→			if !slices.Contains(nodes, scanNode) {
   812→				missingNodes = append(missingNodes, scanNode)
   813→			}
   814→		}
   815→		// Filter nodes by name
   816→		if nodeFilter != "" {
   817→			filteredNodesByName := []string{}
   818→			for _, missingNode := range missingNodes {
   819→				if strings.Contains(missingNode, nodeFilter) {
   820→					filteredNodesByName = append(filteredNodesByName, missingNode)
   821→				}
   822→			}
   823→			missingNodes = filteredNodesByName
   824→		}
   825→		// Sort Missing Nodes Alphanumerically
   826→		slices.Sort(missingNodes)
   827→		// Total Missing
   828→		countNodes = len(missingNodes)
   829→		// Apply paging
   830→		if countNodes > page.ItemsPerPage {
   831→			start := (page.Page - 1) * page.ItemsPerPage
   832→			end := start + page.ItemsPerPage
   833→			if end > countNodes {
   834→				end = countNodes
   835→				hasNextPage = false
   836→			} else {
   837→				hasNextPage = true
   838→			}
   839→			nodes = missingNodes[start:end]
   840→		} else {
   841→			nodes = missingNodes
   842→		}
   843→
   844→	} else {
   845→		// DB Nodes: Count and derive hasNextPage from count
   846→		var cerr error
   847→		countNodes, cerr = r.CountNodes(ctx, queryFilters)
   848→		if cerr != nil {
   849→			cclog.Warn("error while counting node database data (Resolver.NodeMetricsList)")
   850→			return nil, nil, 0, false, cerr
   851→		}
   852→		hasNextPage = page.Page*page.ItemsPerPage < countNodes
   853→	}
   854→
   855→	// Fallback for non-init'd node table in DB; Ignores stateFilter
   856→	if stateFilter == "all" && countNodes == 0 {
   857→		nodes, countNodes, hasNextPage = getNodesFromTopol(cluster, subCluster, nodeFilter, page)
   858→	}
   859→
   860→	return nodes, stateMap, countNodes, hasNextPage, nil
   861→}
   862→
   863→func AccessCheck(ctx context.Context, query sq.SelectBuilder) (sq.SelectBuilder, error) {
   864→	user := GetUserFromContext(ctx)
   865→	return AccessCheckWithUser(user, query)
   866→}
   867→
   868→func AccessCheckWithUser(user *schema.User, query sq.SelectBuilder) (sq.SelectBuilder, error) {
   869→	if user == nil {
   870→		var qnil sq.SelectBuilder
   871→		return qnil, fmt.Errorf("user context is nil")
   872→	}
   873→
   874→	switch {
   875→	// case len(user.Roles) == 1 && user.HasRole(schema.RoleApi): // API-User : Access NodeInfos
   876→	// 	return query, nil
   877→	case user.HasAnyRole([]schema.Role{schema.RoleAdmin, schema.RoleSupport}): // Admin & Support : Access NodeInfos
   878→		return query, nil
   879→	default: // No known Role: No Access, return error
   880→		var qnil sq.SelectBuilder
   881→		return qnil, fmt.Errorf("user has no or unknown roles")
   882→	}
   883→}
   884→
   885→func getNodesFromTopol(cluster string, subCluster string, nodeFilter string, page *model.PageRequest) ([]string, int, bool) {
   886→	// 0) Init additional vars
   887→	hasNextPage := false
   888→	totalNodes := 0
   889→
   890→	// 1) Get list of all nodes
   891→	var topolNodes []string
   892→	if subCluster != "" {
   893→		scNodes := archive.NodeLists[cluster][subCluster]
   894→		topolNodes = scNodes.PrintList()
   895→	} else {
   896→		subClusterNodeLists := archive.NodeLists[cluster]
   897→		for _, nodeList := range subClusterNodeLists {
   898→			topolNodes = append(topolNodes, nodeList.PrintList()...)
   899→		}
   900→	}
   901→
   902→	// 2) Filter nodes
   903→	if nodeFilter != "" {
   904→		filteredNodes := []string{}
   905→		for _, node := range topolNodes {
   906→			if strings.Contains(node, nodeFilter) {
   907→				filteredNodes = append(filteredNodes, node)
   908→			}
   909→		}
   910→		topolNodes = filteredNodes
   911→	}
   912→
   913→	// 2.1) Count total nodes && Sort nodes -> Sorting invalidated after ccms return ...
   914→	totalNodes = len(topolNodes)
   915→	sort.Strings(topolNodes)
   916→
   917→	// 3) Apply paging
   918→	if len(topolNodes) > page.ItemsPerPage {
   919→		start := (page.Page - 1) * page.ItemsPerPage
   920→		end := start + page.ItemsPerPage
   921→		if end >= len(topolNodes) {
   922→			end = len(topolNodes)
   923→			hasNextPage = false
   924→		} else {
   925→			hasNextPage = true
   926→		}
   927→		topolNodes = topolNodes[start:end]
   928→	}
   929→
   930→	return topolNodes, totalNodes, hasNextPage
   931→}
   932→


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→// Package config implements the program configuration data structures, validation and parsing
     7→package config
     8→
     9→import (
    10→	"bytes"
    11→	"encoding/json"
    12→	"time"
    13→
    14→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    15→	"github.com/ClusterCockpit/cc-lib/v2/resampler"
    16→)
    17→
    18→type ProgramConfig struct {
    19→	// Address where the http (or https) server will listen on (for example: 'localhost:80').
    20→	Addr string `json:"addr"`
    21→
    22→	// Addresses from which secured admin API endpoints can be reached, can be wildcard "*"
    23→	APIAllowedIPs []string `json:"api-allowed-ips"`
    24→
    25→	APISubjects *NATSConfig `json:"api-subjects"`
    26→
    27→	// Drop root permissions once .env was read and the port was taken.
    28→	User  string `json:"user"`
    29→	Group string `json:"group"`
    30→
    31→	// Disable authentication (for everything: API, Web-UI, ...)
    32→	DisableAuthentication bool `json:"disable-authentication"`
    33→
    34→	// If `embed-static-files` is true (default), the frontend files are directly
    35→	// embeded into the go binary and expected to be in web/frontend. Only if
    36→	// it is false the files in `static-files` are served instead.
    37→	EmbedStaticFiles bool   `json:"embed-static-files"`
    38→	StaticFiles      string `json:"static-files"`
    39→
    40→	// Path to SQLite database file
    41→	DB string `json:"db"`
    42→
    43→	EnableJobTaggers bool `json:"enable-job-taggers"`
    44→
    45→	// Validate json input against schema
    46→	Validate bool `json:"validate"`
    47→
    48→	// If 0 or empty, the session does not expire!
    49→	SessionMaxAge string `json:"session-max-age"`
    50→
    51→	// If both those options are not empty, use HTTPS using those certificates.
    52→	HTTPSCertFile string `json:"https-cert-file"`
    53→	HTTPSKeyFile  string `json:"https-key-file"`
    54→
    55→	// If not the empty string and `addr` does not end in ":80",
    56→	// redirect every request incoming at port 80 to that url.
    57→	RedirectHTTPTo string `json:"redirect-http-to"`
    58→
    59→	// Where to store MachineState files
    60→	MachineStateDir string `json:"machine-state-dir"`
    61→
    62→	// If not zero, automatically mark jobs as stopped running X seconds longer than their walltime.
    63→	StopJobsExceedingWalltime int `json:"stop-jobs-exceeding-walltime"`
    64→
    65→	// Defines time X in seconds in which jobs are considered to be "short" and will be filtered in specific views.
    66→	ShortRunningJobsDuration int `json:"short-running-jobs-duration"`
    67→
    68→	// Energy Mix CO2 Emission Constant [g/kWh]
    69→	// If entered, displays estimated CO2 emission for job based on jobs totalEnergy
    70→	EmissionConstant int `json:"emission-constant"`
    71→
    72→	// If exists, will enable dynamic zoom in frontend metric plots using the configured values
    73→	EnableResampling *ResampleConfig `json:"resampling"`
    74→
    75→	// Systemd unit name for log viewer (default: "clustercockpit")
    76→	SystemdUnit string `json:"systemd-unit"`
    77→
    78→	// Node state retention configuration
    79→	NodeStateRetention *NodeStateRetention `json:"nodestate-retention"`
    80→
    81→	// Database tuning configuration
    82→	DbConfig *DbConfig `json:"db-config"`
    83→}
    84→
    85→type DbConfig struct {
    86→	CacheSizeMB               int `json:"cache-size-mb"`
    87→	SoftHeapLimitMB           int `json:"soft-heap-limit-mb"`
    88→	MaxOpenConnections        int `json:"max-open-connections"`
    89→	MaxIdleConnections        int `json:"max-idle-connections"`
    90→	ConnectionMaxIdleTimeMins int `json:"max-idle-time-minutes"`
    91→	BusyTimeoutMs             int `json:"busy-timeout-ms"`
    92→}
    93→
    94→type NodeStateRetention struct {
    95→	Policy             string `json:"policy"`      // "delete" or "move"
    96→	Age                int    `json:"age"`         // hours, default 24
    97→	TargetKind         string `json:"target-kind"` // "file" or "s3"
    98→	TargetPath         string `json:"target-path"`
    99→	TargetEndpoint     string `json:"target-endpoint"`
   100→	TargetBucket       string `json:"target-bucket"`
   101→	TargetAccessKey    string `json:"target-access-key"`
   102→	TargetSecretKey    string `json:"target-secret-key"`
   103→	TargetRegion       string `json:"target-region"`
   104→	TargetUsePathStyle bool   `json:"target-use-path-style"`
   105→	MaxFileSizeMB      int    `json:"max-file-size-mb"`
   106→}
   107→
   108→type ResampleConfig struct {
   109→	// Minimum number of points to trigger resampling of data
   110→	MinimumPoints int `json:"minimum-points"`
   111→	// Array of resampling target resolutions, in seconds; Example: [600,300,60]
   112→	Resolutions []int `json:"resolutions"`
   113→	// Trigger next zoom level at less than this many visible datapoints
   114→	Trigger int `json:"trigger"`
   115→}
   116→
   117→type NATSConfig struct {
   118→	SubjectJobEvent  string `json:"subject-job-event"`
   119→	SubjectNodeState string `json:"subject-node-state"`
   120→	JobConcurrency   int    `json:"job-concurrency"`
   121→	NodeConcurrency  int    `json:"node-concurrency"`
   122→}
   123→
   124→type IntRange struct {
   125→	From int `json:"from"`
   126→	To   int `json:"to"`
   127→}
   128→
   129→type TimeRange struct {
   130→	From  *time.Time `json:"from"`
   131→	To    *time.Time `json:"to"`
   132→	Range string     `json:"range,omitempty"`
   133→}
   134→
   135→type FilterRanges struct {
   136→	Duration  *IntRange  `json:"duration"`
   137→	NumNodes  *IntRange  `json:"num-nodes"`
   138→	StartTime *TimeRange `json:"start-time"`
   139→}
   140→
   141→var Keys ProgramConfig = ProgramConfig{
   142→	Addr:                      "localhost:8080",
   143→	EmbedStaticFiles:          true,
   144→	DB:                        "./var/job.db",
   145→	SessionMaxAge:             "168h",
   146→	StopJobsExceedingWalltime: 0,
   147→	ShortRunningJobsDuration:  5 * 60,
   148→}
   149→
   150→func Init(mainConfig json.RawMessage) {
   151→	Validate(configSchema, mainConfig)
   152→	dec := json.NewDecoder(bytes.NewReader(mainConfig))
   153→	dec.DisallowUnknownFields()
   154→	if err := dec.Decode(&Keys); err != nil {
   155→		cclog.Abortf("Config Init: Could not decode config file '%s'.\nError: %s\n", mainConfig, err.Error())
   156→	}
   157→
   158→	if Keys.EnableResampling != nil && Keys.EnableResampling.MinimumPoints > 0 {
   159→		resampler.SetMinimumRequiredPoints(Keys.EnableResampling.MinimumPoints)
   160→	}
   161→}
   162→


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→// Package api provides the REST API layer for ClusterCockpit.
     7→// It handles HTTP requests for job management, user administration,
     8→// cluster queries, node state updates, and metrics storage operations.
     9→// The API supports both JWT token authentication and session-based authentication.
    10→package api
    11→
    12→import (
    13→	"encoding/json"
    14→	"fmt"
    15→	"io"
    16→	"net/http"
    17→	"os"
    18→	"path/filepath"
    19→	"strings"
    20→	"sync"
    21→
    22→	"github.com/ClusterCockpit/cc-backend/internal/auth"
    23→	"github.com/ClusterCockpit/cc-backend/internal/config"
    24→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    25→	"github.com/ClusterCockpit/cc-backend/internal/tagger"
    26→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    27→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    28→	"github.com/ClusterCockpit/cc-lib/v2/util"
    29→	"github.com/go-chi/chi/v5"
    30→)
    31→
    32→// @title                      ClusterCockpit REST API
    33→// @version                    1.0.0
    34→// @description                API for batch job control.
    35→
    36→// @contact.name               ClusterCockpit Project
    37→// @contact.url                https://github.com/ClusterCockpit
    38→// @contact.email              support@clustercockpit.org
    39→
    40→// @license.name               MIT License
    41→// @license.url                https://opensource.org/licenses/MIT
    42→
    43→// @host                       localhost:8080
    44→
    45→// @securityDefinitions.apikey ApiKeyAuth
    46→// @in                         header
    47→// @name                       X-Auth-Token
    48→
    49→const (
    50→	noticeFilePath  = "./var/notice.txt"
    51→	noticeFilePerms = 0o644
    52→	maxNoticeLength = 10000 // Maximum allowed notice content length in characters
    53→)
    54→
    55→type RestAPI struct {
    56→	JobRepository   *repository.JobRepository
    57→	Authentication  *auth.Authentication
    58→	MachineStateDir string
    59→	// RepositoryMutex protects job creation operations from race conditions
    60→	// when checking for duplicate jobs during startJob API calls.
    61→	// It prevents concurrent job starts with the same jobId/cluster/startTime
    62→	// from creating duplicate entries in the database.
    63→	RepositoryMutex sync.Mutex
    64→}
    65→
    66→// New creates and initializes a new RestAPI instance with configured dependencies.
    67→func New() *RestAPI {
    68→	return &RestAPI{
    69→		JobRepository:   repository.GetJobRepository(),
    70→		MachineStateDir: config.Keys.MachineStateDir,
    71→		Authentication:  auth.GetAuthInstance(),
    72→	}
    73→}
    74→
    75→// MountAPIRoutes registers REST API endpoints for job and cluster management.
    76→// These routes use JWT token authentication via the X-Auth-Token header.
    77→func (api *RestAPI) MountAPIRoutes(r chi.Router) {
    78→	// REST API Uses TokenAuth
    79→	// User List
    80→	r.Get("/users/", api.getUsers)
    81→	// Cluster List
    82→	r.Get("/clusters/", api.getClusters)
    83→	// Slurm node state
    84→	r.Post("/nodestate/", api.updateNodeStates)
    85→	r.Put("/nodestate/", api.updateNodeStates)
    86→	// Job Handler
    87→	if config.Keys.APISubjects == nil {
    88→		cclog.Info("Enabling REST start/stop job API")
    89→		r.Post("/jobs/start_job/", api.startJob)
    90→		r.Put("/jobs/start_job/", api.startJob)
    91→		r.Post("/jobs/stop_job/", api.stopJobByRequest)
    92→		r.Put("/jobs/stop_job/", api.stopJobByRequest)
    93→	}
    94→	r.Get("/jobs/", api.getJobs)
    95→	r.Get("/jobs/used_nodes", api.getUsedNodes)
    96→	r.Post("/jobs/tag_job/{id}", api.tagJob)
    97→	r.Patch("/jobs/tag_job/{id}", api.tagJob)
    98→	r.Delete("/jobs/tag_job/{id}", api.removeTagJob)
    99→	r.Patch("/jobs/edit_meta/{id}", api.editMeta)
   100→	r.Patch("/jobs/edit_meta/", api.editMetaByRequest)