<local-command-caveat>Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.</local-command-caveat>


<command-name>/model</command-name>
            <command-message>model</command-message>
            <command-args></command-args>


<local-command-stdout>Set model to Default (Sonnet 4.6 · Best for everyday tasks)</local-command-stdout>


What is the default for EnableJobTaggers if the option is not set in the config file?


internal/config/config.go:43:	EnableJobTaggers bool `json:"enable-job-taggers"`


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→// Package config implements the program configuration data structures, validation and parsing
     7→package config
     8→
     9→import (
    10→	"bytes"
    11→	"encoding/json"
    12→	"time"
    13→
    14→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    15→	"github.com/ClusterCockpit/cc-lib/v2/resampler"
    16→)
    17→
    18→type ProgramConfig struct {
    19→	// Address where the http (or https) server will listen on (for example: 'localhost:80').
    20→	Addr string `json:"addr"`
    21→
    22→	// Addresses from which secured admin API endpoints can be reached, can be wildcard "*"
    23→	APIAllowedIPs []string `json:"api-allowed-ips"`
    24→
    25→	APISubjects *NATSConfig `json:"api-subjects"`
    26→
    27→	// Drop root permissions once .env was read and the port was taken.
    28→	User  string `json:"user"`
    29→	Group string `json:"group"`
    30→
    31→	// Disable authentication (for everything: API, Web-UI, ...)
    32→	DisableAuthentication bool `json:"disable-authentication"`
    33→
    34→	// If `embed-static-files` is true (default), the frontend files are directly
    35→	// embeded into the go binary and expected to be in web/frontend. Only if
    36→	// it is false the files in `static-files` are served instead.
    37→	EmbedStaticFiles bool   `json:"embed-static-files"`
    38→	StaticFiles      string `json:"static-files"`
    39→
    40→	// Path to SQLite database file
    41→	DB string `json:"db"`
    42→
    43→	EnableJobTaggers bool `json:"enable-job-taggers"`
    44→
    45→	// Validate json input against schema
    46→	Validate bool `json:"validate"`
    47→
    48→	// If 0 or empty, the session does not expire!
    49→	SessionMaxAge string `json:"session-max-age"`
    50→
    51→	// If both those options are not empty, use HTTPS using those certificates.
    52→	HTTPSCertFile string `json:"https-cert-file"`
    53→	HTTPSKeyFile  string `json:"https-key-file"`
    54→
    55→	// If not the empty string and `addr` does not end in ":80",
    56→	// redirect every request incoming at port 80 to that url.
    57→	RedirectHTTPTo string `json:"redirect-http-to"`
    58→
    59→	// Where to store MachineState files
    60→	MachineStateDir string `json:"machine-state-dir"`
    61→
    62→	// If not zero, automatically mark jobs as stopped running X seconds longer than their walltime.
    63→	StopJobsExceedingWalltime int `json:"stop-jobs-exceeding-walltime"`
    64→
    65→	// Defines time X in seconds in which jobs are considered to be "short" and will be filtered in specific views.
    66→	ShortRunningJobsDuration int `json:"short-running-jobs-duration"`
    67→
    68→	// Energy Mix CO2 Emission Constant [g/kWh]
    69→	// If entered, displays estimated CO2 emission for job based on jobs totalEnergy
    70→	EmissionConstant int `json:"emission-constant"`
    71→
    72→	// If exists, will enable dynamic zoom in frontend metric plots using the configured values
    73→	EnableResampling *ResampleConfig `json:"resampling"`
    74→
    75→	// Systemd unit name for log viewer (default: "clustercockpit")
    76→	SystemdUnit string `json:"systemd-unit"`
    77→
    78→	// Node state retention configuration
    79→	NodeStateRetention *NodeStateRetention `json:"nodestate-retention"`
    80→}

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

// Package config implements the program configuration data structures, validation and parsing
package config

import (
	"bytes"
	"encoding/json"
	"time"

	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/resampler"
)

type ProgramConfig struct {
	// Address where the http (or https) server will listen on (for example: 'localhost:80').
	Addr string `json:"addr"`

	// Addresses from which secured admin API endpoints can be reached, can be wildcard "*"
	APIAllowedIPs []string `json:"api-allowed-ips"`

	APISubjects *NATSConfig `json:"api-subjects"`

	// Drop root permissions once .env was read and the port was taken.
	User  string `json:"user"`
	Group string `json:"group"`

	// Disable authentication (for everything: API, Web-UI, ...)
	DisableAuthentication bool `json:"disable-authentication"`

	// If `embed-static-files` is true (default), the frontend files are directly
	// embeded into the go binary and expected to be in web/frontend. Only if
	// it is false the files in `static-files` are served instead.
	EmbedStaticFiles bool   `json:"embed-static-files"`
	StaticFiles      string `json:"static-files"`

	// Path to SQLite database file
	DB string `json:"db"`

	EnableJobTaggers bool `json:"enable-job-taggers"`

	// Validate json input against schema
	Validate bool `json:"validate"`

	// If 0 or empty, the session does not expire!
	SessionMaxAge string `json:"session-max-age"`

	// If both those options are not empty, use HTTPS using those certificates.
	HTTPSCertFile string `json:"https-cert-file"`
	HTTPSKeyFile  string `json:"https-key-file"`

	// If not the empty string and `addr` does not end in ":80",
	// redirect every request incoming at port 80 to that url.
	RedirectHTTPTo string `json:"redirect-http-to"`

	// Where to store MachineState files
	MachineStateDir string `json:"machine-state-dir"`

	// If not zero, automatically mark jobs as stopped running X seconds longer than their walltime.
	StopJobsExceedingWalltime int `json:"stop-jobs-exceeding-walltime"`

	// Defines time X in seconds in which jobs are considered to be "short" and will be filtered in specific views.
	ShortRunningJobsDuration int `json:"short-running-jobs-duration"`

	// Energy Mix CO2 Emission Constant [g/kWh]
	// If entered, displays estimated CO2 emission for job based on jobs totalEnergy
	EmissionConstant int `json:"emission-constant"`

	// If exists, will enable dynamic zoom in frontend metric plots using the configured values
	EnableResampling *ResampleConfig `json:"resampling"`

	// Systemd unit name for log viewer (default: "clustercockpit")
	SystemdUnit string `json:"systemd-unit"`

	// Node state retention configuration
	NodeStateRetention *NodeStateRetention `json:"nodestate-retention"`
}


The default is `false`. `EnableJobTaggers` is a plain `bool` field with no explicit default set — when not present in the config JSON, Go's zero value for `bool` applies, which is `false`.


Currently a segvault occurs when the option enable-job-taggers is set to true but the tagger rules directories are missing. Analyse the problem and fix it.


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→// Package tagger provides automatic job tagging functionality for cc-backend.
     7→// It supports detecting applications and classifying jobs based on configurable rules.
     8→// Tags are automatically applied when jobs start or stop, or can be applied retroactively
     9→// to existing jobs using RunTaggers.
    10→package tagger
    11→
    12→import (
    13→	"fmt"
    14→	"sync"
    15→
    16→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    17→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    18→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    19→)
    20→
    21→// Tagger is the interface that must be implemented by all tagging components.
    22→// Taggers can be registered at job start or stop events to automatically apply tags.
    23→type Tagger interface {
    24→	// Register initializes the tagger and loads any required configuration.
    25→	// It should be called once before the tagger is used.
    26→	Register() error
    27→
    28→	// Match evaluates the tagger's rules against a job and applies appropriate tags.
    29→	// It is called for each job that needs to be evaluated.
    30→	Match(job *schema.Job)
    31→}
    32→
    33→// TaggerInfo holds metadata about a tagger for JSON serialization.
    34→type TaggerInfo struct {
    35→	Name    string `json:"name"`
    36→	Type    string `json:"type"`
    37→	Running bool   `json:"running"`
    38→}
    39→
    40→var (
    41→	initOnce     sync.Once
    42→	jobTagger    *JobTagger
    43→	statusMu     sync.Mutex
    44→	taggerStatus = map[string]bool{}
    45→)
    46→
    47→// Known tagger definitions: name -> (type, factory)
    48→type taggerDef struct {
    49→	ttype   string
    50→	factory func() Tagger
    51→}
    52→
    53→var knownTaggers = map[string]taggerDef{
    54→	"AppTagger":      {ttype: "start", factory: func() Tagger { return &AppTagger{} }},
    55→	"JobClassTagger": {ttype: "stop", factory: func() Tagger { return &JobClassTagger{} }},
    56→}
    57→
    58→// JobTagger coordinates multiple taggers that run at different job lifecycle events.
    59→// It maintains separate lists of taggers that run when jobs start and when they stop.
    60→type JobTagger struct {
    61→	// startTaggers are applied when a job starts (e.g., application detection)
    62→	startTaggers []Tagger
    63→	// stopTaggers are applied when a job completes (e.g., job classification)
    64→	stopTaggers []Tagger
    65→}
    66→
    67→func newTagger() {
    68→	jobTagger = &JobTagger{}
    69→	jobTagger.startTaggers = make([]Tagger, 0)
    70→	jobTagger.startTaggers = append(jobTagger.startTaggers, &AppTagger{})
    71→	jobTagger.stopTaggers = make([]Tagger, 0)
    72→	jobTagger.stopTaggers = append(jobTagger.stopTaggers, &JobClassTagger{})
    73→
    74→	for _, tagger := range jobTagger.startTaggers {
    75→		if err := tagger.Register(); err != nil {
    76→			cclog.Errorf("failed to register start tagger: %s", err)
    77→		}
    78→	}
    79→	for _, tagger := range jobTagger.stopTaggers {
    80→		if err := tagger.Register(); err != nil {
    81→			cclog.Errorf("failed to register stop tagger: %s", err)
    82→		}
    83→	}
    84→}
    85→
    86→// Init initializes the job tagger system and registers it with the job repository.
    87→// This function is safe to call multiple times; initialization only occurs once.
    88→// It should be called during application startup.
    89→func Init() {
    90→	initOnce.Do(func() {
    91→		newTagger()
    92→		repository.RegisterJobHook(jobTagger)
    93→	})
    94→}
    95→
    96→// JobStartCallback is called when a job starts.
    97→// It runs all registered start taggers (e.g., application detection) on the job.
    98→func (jt *JobTagger) JobStartCallback(job *schema.Job) {
    99→	for _, tagger := range jt.startTaggers {
   100→		tagger.Match(job)
   101→	}
   102→}
   103→
   104→// JobStopCallback is called when a job completes.
   105→// It runs all registered stop taggers (e.g., job classification) on the job.
   106→func (jt *JobTagger) JobStopCallback(job *schema.Job) {
   107→	for _, tagger := range jt.stopTaggers {
   108→		tagger.Match(job)
   109→	}
   110→}
   111→
   112→// ListTaggers returns information about all known taggers with their current running status.
   113→func ListTaggers() []TaggerInfo {
   114→	statusMu.Lock()
   115→	defer statusMu.Unlock()
   116→
   117→	result := make([]TaggerInfo, 0, len(knownTaggers))
   118→	for name, def := range knownTaggers {
   119→		result = append(result, TaggerInfo{
   120→			Name:    name,
   121→			Type:    def.ttype,
   122→			Running: taggerStatus[name],
   123→		})
   124→	}
   125→	return result
   126→}
   127→
   128→// RunTaggerByName starts a tagger by name asynchronously on all jobs.
   129→// Returns an error if the name is unknown or the tagger is already running.
   130→func RunTaggerByName(name string) error {
   131→	def, ok := knownTaggers[name]
   132→	if !ok {
   133→		return fmt.Errorf("unknown tagger: %s", name)
   134→	}
   135→
   136→	statusMu.Lock()
   137→	if taggerStatus[name] {
   138→		statusMu.Unlock()
   139→		return fmt.Errorf("tagger %s is already running", name)
   140→	}
   141→	taggerStatus[name] = true
   142→	statusMu.Unlock()
   143→
   144→	go func() {
   145→		defer func() {
   146→			statusMu.Lock()
   147→			taggerStatus[name] = false
   148→			statusMu.Unlock()
   149→		}()
   150→
   151→		t := def.factory()
   152→		if err := t.Register(); err != nil {
   153→			cclog.Errorf("Failed to register tagger %s: %s", name, err)
   154→			return
   155→		}
   156→
   157→		r := repository.GetJobRepository()
   158→		jl, err := r.GetJobList(0, 0)
   159→		if err != nil {
   160→			cclog.Errorf("Error getting job list for tagger %s: %s", name, err)
   161→			return
   162→		}
   163→
   164→		cclog.Infof("Running tagger %s on %d jobs", name, len(jl))
   165→		for _, id := range jl {
   166→			job, err := r.FindByIDDirect(id)
   167→			if err != nil {
   168→				cclog.Errorf("Error getting job %d for tagger %s: %s", id, name, err)
   169→				continue
   170→			}
   171→			t.Match(job)
   172→		}
   173→		cclog.Infof("Tagger %s completed", name)
   174→	}()
   175→
   176→	return nil
   177→}
   178→
   179→// RunTaggers applies all configured taggers to all existing jobs in the repository.
   180→// This is useful for retroactively applying tags to jobs that were created before
   181→// the tagger system was initialized or when new tagging rules are added.
   182→// It fetches all jobs and runs both start and stop taggers on each one.
   183→func RunTaggers() error {
   184→	newTagger()
   185→	r := repository.GetJobRepository()
   186→	jl, err := r.GetJobList(0, 0) // 0 limit means get all jobs (no pagination)
   187→	if err != nil {
   188→		cclog.Errorf("Error while getting job list %s", err)
   189→		return err
   190→	}
   191→
   192→	for _, id := range jl {
   193→		job, err := r.FindByIDDirect(id)
   194→		if err != nil {
   195→			cclog.Errorf("Error while getting job %s", err)
   196→			return err
   197→		}
   198→		for _, tagger := range jobTagger.startTaggers {
   199→			tagger.Match(job)
   200→		}
   201→		for _, tagger := range jobTagger.stopTaggers {
   202→			cclog.Infof("Run stop tagger for job %d", *job.ID)
   203→			tagger.Match(job)
   204→		}
   205→	}
   206→	return nil
   207→}
   208→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

// Package tagger provides automatic job tagging functionality for cc-backend.
// It supports detecting applications and classifying jobs based on configurable rules.
// Tags are automatically applied when jobs start or stop, or can be applied retroactively
// to existing jobs using RunTaggers.
package tagger

import (
	"fmt"
	"sync"

	"github.com/ClusterCockpit/cc-backend/internal/repository"
	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/schema"
)

// Tagger is the interface that must be implemented by all tagging components.
// Taggers can be registered at job start or stop events to automatically apply tags.
type Tagger interface {
	// Register initializes the tagger and loads any required configuration.
	// It should be called once before the tagger is used.
	Register() error

	// Match evaluates the tagger's rules against a job and applies appropriate tags.
	// It is called for each job that needs to be evaluated.
	Match(job *schema.Job)
}

// TaggerInfo holds metadata about a tagger for JSON serialization.
type TaggerInfo struct {
	Name    string `json:"name"`
	Type    string `json:"type"`
	Running bool   `json:"running"`
}

var (
	initOnce     sync.Once
	jobTagger    *JobTagger
	statusMu     sync.Mutex
	taggerStatus = map[string]bool{}
)

// Known tagger definitions: name -> (type, factory)
type taggerDef struct {
	ttype   string
	factory func() Tagger
}

var knownTaggers = map[string]taggerDef{
	"AppTagger":      {ttype: "start", factory: func() Tagger { return &AppTagger{} }},
	"JobClassTagger": {ttype: "stop", factory: func() Tagger { return &JobClassTagger{} }},
}

// JobTagger coordinates multiple taggers that run at different job lifecycle events.
// It maintains separate lists of taggers that run when jobs start and when they stop.
type JobTagger struct {
	// startTaggers are applied when a job starts (e.g., application detection)
	startTaggers []Tagger
	// stopTaggers are applied when a job completes (e.g., job classification)
	stopTaggers []Tagger
}

func newTagger() {
	jobTagger = &JobTagger{}
	jobTagger.startTaggers = make([]Tagger, 0)
	jobTagger.startTaggers = append(jobTagger.startTaggers, &AppTagger{})
	jobTagger.stopTaggers = make([]Tagger, 0)
	jobTagger.stopTaggers = append(jobTagger.stopTaggers, &JobClassTagger{})

	for _, tagger := range jobTagger.startTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register start tagger: %s", err)
		}
	}
	for _, tagger := range jobTagger.stopTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register stop tagger: %s", err)
		}
	}
}

// Init initializes the job tagger system and registers it with the job repository.
// This function is safe to call multiple times; initialization only occurs once.
// It should be called during application startup.
func Init() {
	initOnce.Do(func() {
		newTagger()
		repository.RegisterJobHook(jobTagger)
	})
}

// JobStartCallback is called when a job starts.
// It runs all registered start taggers (e.g., application detection) on the job.
func (jt *JobTagger) JobStartCallback(job *schema.Job) {
	for _, tagger := range jt.startTaggers {
		tagger.Match(job)
	}
}

// JobStopCallback is called when a job completes.
// It runs all registered stop taggers (e.g., job classification) on the job.
func (jt *JobTagger) JobStopCallback(job *schema.Job) {
	for _, tagger := range jt.stopTaggers {
		tagger.Match(job)
	}
}

// ListTaggers returns information about all known taggers with their current running status.
func ListTaggers() []TaggerInfo {
	statusMu.Lock()
	defer statusMu.Unlock()

	result := make([]TaggerInfo, 0, len(knownTaggers))
	for name, def := range knownTaggers {
		result = append(result, TaggerInfo{
			Name:    name,
			Type:    def.ttype,
			Running: taggerStatus[name],
		})
	}
	return result
}

// RunTaggerByName starts a tagger by name asynchronously on all jobs.
// Returns an error if the name is unknown or the tagger is already running.
func RunTaggerByName(name string) error {
	def, ok := knownTaggers[name]
	if !ok {
		return fmt.Errorf("unknown tagger: %s", name)
	}

	statusMu.Lock()
	if taggerStatus[name] {
		statusMu.Unlock()
		return fmt.Errorf("tagger %s is already running", name)
	}
	taggerStatus[name] = true
	statusMu.Unlock()

	go func() {
		defer func() {
			statusMu.Lock()
			taggerStatus[name] = false
			statusMu.Unlock()
		}()

		t := def.factory()
		if err := t.Register(); err != nil {
			cclog.Errorf("Failed to register tagger %s: %s", name, err)
			return
		}

		r := repository.GetJobRepository()
		jl, err := r.GetJobList(0, 0)
		if err != nil {
			cclog.Errorf("Error getting job list for tagger %s: %s", name, err)
			return
		}

		cclog.Infof("Running tagger %s on %d jobs", name, len(jl))
		for _, id := range jl {
			job, err := r.FindByIDDirect(id)
			if err != nil {
				cclog.Errorf("Error getting job %d for tagger %s: %s", id, name, err)
				continue
			}
			t.Match(job)
		}
		cclog.Infof("Tagger %s completed", name)
	}()

	return nil
}

// RunTaggers applies all configured taggers to all existing jobs in the repository.
// This is useful for retroactively applying tags to jobs that were created before
// the tagger system was initialized or when new tagging rules are added.
// It fetches all jobs and runs both start and stop taggers on each one.
func RunTaggers() error {
	newTagger()
	r := repository.GetJobRepository()
	jl, err := r.GetJobList(0, 0) // 0 limit means get all jobs (no pagination)
	if err != nil {
		cclog.Errorf("Error while getting job list %s", err)
		return err
	}

	for _, id := range jl {
		job, err := r.FindByIDDirect(id)
		if err != nil {
			cclog.Errorf("Error while getting job %s", err)
			return err
		}
		for _, tagger := range jobTagger.startTaggers {
			tagger.Match(job)
		}
		for _, tagger := range jobTagger.stopTaggers {
			cclog.Infof("Run stop tagger for job %d", *job.ID)
			tagger.Match(job)
		}
	}
	return nil
}


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package tagger
     7→
     8→import (
     9→	"bufio"
    10→	"fmt"
    11→	"os"
    12→	"path/filepath"
    13→	"regexp"
    14→	"strings"
    15→
    16→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    17→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    18→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    19→	"github.com/ClusterCockpit/cc-lib/v2/util"
    20→)
    21→
    22→func metadataKeys(m map[string]string) []string {
    23→	keys := make([]string, 0, len(m))
    24→	for k := range m {
    25→		keys = append(keys, k)
    26→	}
    27→	return keys
    28→}
    29→
    30→const (
    31→	// defaultConfigPath is the default path for application tagging configuration
    32→	defaultConfigPath = "./var/tagger/apps"
    33→	// tagTypeApp is the tag type identifier for application tags
    34→	tagTypeApp = "app"
    35→	// configDirMatch is the directory name used for matching filesystem events
    36→	configDirMatch = "apps"
    37→)
    38→
    39→type appInfo struct {
    40→	tag      string
    41→	patterns []*regexp.Regexp
    42→}
    43→
    44→// AppTagger detects applications by matching patterns in job scripts.
    45→// It loads application patterns from an external configuration directory and can dynamically reload
    46→// configuration when files change. When a job script matches a pattern,
    47→// the corresponding application tag is automatically applied.
    48→type AppTagger struct {
    49→	// apps holds application patterns in deterministic order
    50→	apps []appInfo
    51→	// tagType is the type of tag ("app")
    52→	tagType string
    53→	// cfgPath is the path to watch for configuration changes
    54→	cfgPath string
    55→}
    56→
    57→func (t *AppTagger) scanApp(f *os.File, fns string) {
    58→	scanner := bufio.NewScanner(f)
    59→	tag := strings.TrimSuffix(fns, filepath.Ext(fns))
    60→	ai := appInfo{tag: tag, patterns: make([]*regexp.Regexp, 0)}
    61→
    62→	for scanner.Scan() {
    63→		line := strings.TrimSpace(scanner.Text())
    64→		if line == "" {
    65→			continue
    66→		}
    67→		// Wrap pattern to skip comment lines: match only if not preceded by # on the same line
    68→		wrapped := `(?m)^[^#]*` + line
    69→		re, err := regexp.Compile(wrapped)
    70→		if err != nil {
    71→			cclog.Errorf("invalid regex pattern '%s' (wrapped: '%s') in %s: %v", line, wrapped, fns, err)
    72→			continue
    73→		}
    74→		ai.patterns = append(ai.patterns, re)
    75→	}
    76→
    77→	// Remove existing entry for this tag if present
    78→	for i, a := range t.apps {
    79→		if a.tag == tag {
    80→			t.apps = append(t.apps[:i], t.apps[i+1:]...)
    81→			break
    82→		}
    83→	}
    84→
    85→	cclog.Infof("AppTagger loaded %d patterns for %s", len(ai.patterns), tag)
    86→	t.apps = append(t.apps, ai)
    87→}
    88→
    89→// EventMatch checks if a filesystem event should trigger configuration reload.
    90→// It returns true if the event path contains "apps".
    91→func (t *AppTagger) EventMatch(s string) bool {
    92→	return strings.Contains(s, configDirMatch)
    93→}
    94→
    95→// EventCallback is triggered when the configuration directory changes.
    96→// It reloads all application pattern files from the watched directory.
    97→func (t *AppTagger) EventCallback() {
    98→	files, err := os.ReadDir(t.cfgPath)
    99→	if err != nil {
   100→		cclog.Fatal(err)
   101→	}
   102→
   103→	t.apps = make([]appInfo, 0)
   104→
   105→	for _, fn := range files {
   106→		if fn.IsDir() {
   107→			continue
   108→		}
   109→		fns := fn.Name()
   110→		cclog.Debugf("Process: %s", fns)
   111→		f, err := os.Open(filepath.Join(t.cfgPath, fns))
   112→		if err != nil {
   113→			cclog.Errorf("error opening app file %s: %#v", fns, err)
   114→			continue
   115→		}
   116→		t.scanApp(f, fns)
   117→		if err := f.Close(); err != nil {
   118→			cclog.Errorf("error closing app file %s: %#v", fns, err)
   119→		}
   120→	}
   121→}
   122→
   123→// Register initializes the AppTagger by loading application patterns from external folder.
   124→// It sets up a file watch on ./var/tagger/apps if it exists, allowing for
   125→// dynamic configuration updates without restarting the application.
   126→// Returns an error if the configuration path does not exist or cannot be read.
   127→func (t *AppTagger) Register() error {
   128→	if t.cfgPath == "" {
   129→		t.cfgPath = defaultConfigPath
   130→	}
   131→	t.tagType = tagTypeApp
   132→	t.apps = make([]appInfo, 0)
   133→
   134→	if !util.CheckFileExists(t.cfgPath) {
   135→		return fmt.Errorf("configuration path does not exist: %s", t.cfgPath)
   136→	}
   137→
   138→	files, err := os.ReadDir(t.cfgPath)
   139→	if err != nil {
   140→		return fmt.Errorf("error reading app folder: %#v", err)
   141→	}
   142→
   143→	for _, fn := range files {
   144→		if fn.IsDir() {
   145→			continue
   146→		}
   147→		fns := fn.Name()
   148→		cclog.Debugf("Process: %s", fns)
   149→		f, err := os.Open(filepath.Join(t.cfgPath, fns))
   150→		if err != nil {
   151→			cclog.Errorf("error opening app file %s: %#v", fns, err)
   152→			continue
   153→		}
   154→		t.scanApp(f, fns)
   155→		if err := f.Close(); err != nil {
   156→			cclog.Errorf("error closing app file %s: %#v", fns, err)
   157→		}
   158→	}
   159→
   160→	cclog.Infof("Setup file watch for %s", t.cfgPath)
   161→	util.AddListener(t.cfgPath, t)
   162→
   163→	return nil
   164→}
   165→
   166→// Match attempts to detect the application used by a job by analyzing its job script.
   167→// It fetches the job metadata, extracts the job script, and matches it against
   168→// all configured application patterns using regular expressions.
   169→// If a match is found, the corresponding application tag is added to the job.
   170→// Multiple application tags can be applied if patterns for different apps match.
   171→func (t *AppTagger) Match(job *schema.Job) {
   172→	r := repository.GetJobRepository()
   173→
   174→	if len(t.apps) == 0 {
   175→		cclog.Warn("AppTagger: no app patterns loaded, skipping match")
   176→		return
   177→	}
   178→
   179→	metadata, err := r.FetchMetadata(job)
   180→	if err != nil {
   181→		cclog.Debugf("AppTagger: cannot fetch metadata for job %d on %s: %v", job.JobID, job.Cluster, err)
   182→		return
   183→	}
   184→
   185→	if metadata == nil {
   186→		cclog.Debugf("AppTagger: metadata is nil for job %d on %s", job.JobID, job.Cluster)
   187→		return
   188→	}
   189→
   190→	jobscript, ok := metadata["jobScript"]
   191→	if !ok {
   192→		cclog.Debugf("AppTagger: no 'jobScript' key in metadata for job %d on %s (keys: %v)",
   193→			job.JobID, job.Cluster, metadataKeys(metadata))
   194→		return
   195→	}
   196→
   197→	if len(jobscript) == 0 {
   198→		cclog.Debugf("AppTagger: empty jobScript for job %d on %s", job.JobID, job.Cluster)
   199→		return
   200→	}
   201→
   202→	id := *job.ID
   203→	jobscriptLower := strings.ToLower(jobscript)
   204→	cclog.Debugf("AppTagger: matching job %d (script length: %d) against %d apps", id, len(jobscriptLower), len(t.apps))
   205→
   206→	matched := false
   207→	for _, a := range t.apps {
   208→		for _, re := range a.patterns {
   209→			if re.MatchString(jobscriptLower) {
   210→				if r.HasTag(id, t.tagType, a.tag) {
   211→					cclog.Debugf("AppTagger: job %d already has tag %s:%s, skipping", id, t.tagType, a.tag)
   212→				} else {
   213→					cclog.Debugf("AppTagger: pattern '%s' matched for app '%s' on job %d", re.String(), a.tag, id)
   214→					if _, err := r.AddTagOrCreateDirect(id, t.tagType, a.tag); err != nil {
   215→						cclog.Errorf("AppTagger: failed to add tag '%s' to job %d: %v", a.tag, id, err)
   216→					}
   217→				}
   218→				matched = true
   219→				break // matched this app, move to next app
   220→			}
   221→		}
   222→	}
   223→
   224→	if !matched {
   225→		cclog.Debugf("AppTagger: no pattern matched for job %d on %s", id, job.Cluster)
   226→	}
   227→}
   228→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package tagger

import (
	"bufio"
	"fmt"
	"os"
	"path/filepath"
	"regexp"
	"strings"

	"github.com/ClusterCockpit/cc-backend/internal/repository"
	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/schema"
	"github.com/ClusterCockpit/cc-lib/v2/util"
)

func metadataKeys(m map[string]string) []string {
	keys := make([]string, 0, len(m))
	for k := range m {
		keys = append(keys, k)
	}
	return keys
}

const (
	// defaultConfigPath is the default path for application tagging configuration
	defaultConfigPath = "./var/tagger/apps"
	// tagTypeApp is the tag type identifier for application tags
	tagTypeApp = "app"
	// configDirMatch is the directory name used for matching filesystem events
	configDirMatch = "apps"
)

type appInfo struct {
	tag      string
	patterns []*regexp.Regexp
}

// AppTagger detects applications by matching patterns in job scripts.
// It loads application patterns from an external configuration directory and can dynamically reload
// configuration when files change. When a job script matches a pattern,
// the corresponding application tag is automatically applied.
type AppTagger struct {
	// apps holds application patterns in deterministic order
	apps []appInfo
	// tagType is the type of tag ("app")
	tagType string
	// cfgPath is the path to watch for configuration changes
	cfgPath string
}

func (t *AppTagger) scanApp(f *os.File, fns string) {
	scanner := bufio.NewScanner(f)
	tag := strings.TrimSuffix(fns, filepath.Ext(fns))
	ai := appInfo{tag: tag, patterns: make([]*regexp.Regexp, 0)}

	for scanner.Scan() {
		line := strings.TrimSpace(scanner.Text())
		if line == "" {
			continue
		}
		// Wrap pattern to skip comment lines: match only if not preceded by # on the same line
		wrapped := `(?m)^[^#]*` + line
		re, err := regexp.Compile(wrapped)
		if err != nil {
			cclog.Errorf("invalid regex pattern '%s' (wrapped: '%s') in %s: %v", line, wrapped, fns, err)
			continue
		}
		ai.patterns = append(ai.patterns, re)
	}

	// Remove existing entry for this tag if present
	for i, a := range t.apps {
		if a.tag == tag {
			t.apps = append(t.apps[:i], t.apps[i+1:]...)
			break
		}
	}

	cclog.Infof("AppTagger loaded %d patterns for %s", len(ai.patterns), tag)
	t.apps = append(t.apps, ai)
}

// EventMatch checks if a filesystem event should trigger configuration reload.
// It returns true if the event path contains "apps".
func (t *AppTagger) EventMatch(s string) bool {
	return strings.Contains(s, configDirMatch)
}

// EventCallback is triggered when the configuration directory changes.
// It reloads all application pattern files from the watched directory.
func (t *AppTagger) EventCallback() {
	files, err := os.ReadDir(t.cfgPath)
	if err != nil {
		cclog.Fatal(err)
	}

	t.apps = make([]appInfo, 0)

	for _, fn := range files {
		if fn.IsDir() {
			continue
		}
		fns := fn.Name()
		cclog.Debugf("Process: %s", fns)
		f, err := os.Open(filepath.Join(t.cfgPath, fns))
		if err != nil {
			cclog.Errorf("error opening app file %s: %#v", fns, err)
			continue
		}
		t.scanApp(f, fns)
		if err := f.Close(); err != nil {
			cclog.Errorf("error closing app file %s: %#v", fns, err)
		}
	}
}

// Register initializes the AppTagger by loading application patterns from external folder.
// It sets up a file watch on ./var/tagger/apps if it exists, allowing for
// dynamic configuration updates without restarting the application.
// Returns an error if the configuration path does not exist or cannot be read.
func (t *AppTagger) Register() error {
	if t.cfgPath == "" {
		t.cfgPath = defaultConfigPath
	}
	t.tagType = tagTypeApp
	t.apps = make([]appInfo, 0)

	if !util.CheckFileExists(t.cfgPath) {
		return fmt.Errorf("configuration path does not exist: %s", t.cfgPath)
	}

	files, err := os.ReadDir(t.cfgPath)
	if err != nil {
		return fmt.Errorf("error reading app folder: %#v", err)
	}

	for _, fn := range files {
		if fn.IsDir() {
			continue
		}
		fns := fn.Name()
		cclog.Debugf("Process: %s", fns)
		f, err := os.Open(filepath.Join(t.cfgPath, fns))
		if err != nil {
			cclog.Errorf("error opening app file %s: %#v", fns, err)
			continue
		}
		t.scanApp(f, fns)
		if err := f.Close(); err != nil {
			cclog.Errorf("error closing app file %s: %#v", fns, err)
		}
	}

	cclog.Infof("Setup file watch for %s", t.cfgPath)
	util.AddListener(t.cfgPath, t)

	return nil
}

// Match attempts to detect the application used by a job by analyzing its job script.
// It fetches the job metadata, extracts the job script, and matches it against
// all configured application patterns using regular expressions.
// If a match is found, the corresponding application tag is added to the job.
// Multiple application tags can be applied if patterns for different apps match.
func (t *AppTagger) Match(job *schema.Job) {
	r := repository.GetJobRepository()

	if len(t.apps) == 0 {
		cclog.Warn("AppTagger: no app patterns loaded, skipping match")
		return
	}

	metadata, err := r.FetchMetadata(job)
	if err != nil {
		cclog.Debugf("AppTagger: cannot fetch metadata for job %d on %s: %v", job.JobID, job.Cluster, err)
		return
	}

	if metadata == nil {
		cclog.Debugf("AppTagger: metadata is nil for job %d on %s", job.JobID, job.Cluster)
		return
	}

	jobscript, ok := metadata["jobScript"]
	if !ok {
		cclog.Debugf("AppTagger: no 'jobScript' key in metadata for job %d on %s (keys: %v)",
			job.JobID, job.Cluster, metadataKeys(metadata))
		return
	}

	if len(jobscript) == 0 {
		cclog.Debugf("AppTagger: empty jobScript for job %d on %s", job.JobID, job.Cluster)
		return
	}

	id := *job.ID
	jobscriptLower := strings.ToLower(jobscript)
	cclog.Debugf("AppTagger: matching job %d (script length: %d) against %d apps", id, len(jobscriptLower), len(t.apps))

	matched := false
	for _, a := range t.apps {
		for _, re := range a.patterns {
			if re.MatchString(jobscriptLower) {
				if r.HasTag(id, t.tagType, a.tag) {
					cclog.Debugf("AppTagger: job %d already has tag %s:%s, skipping", id, t.tagType, a.tag)
				} else {
					cclog.Debugf("AppTagger: pattern '%s' matched for app '%s' on job %d", re.String(), a.tag, id)
					if _, err := r.AddTagOrCreateDirect(id, t.tagType, a.tag); err != nil {
						cclog.Errorf("AppTagger: failed to add tag '%s' to job %d: %v", a.tag, id, err)
					}
				}
				matched = true
				break // matched this app, move to next app
			}
		}
	}

	if !matched {
		cclog.Debugf("AppTagger: no pattern matched for job %d on %s", id, job.Cluster)
	}
}


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package tagger
     7→
     8→import (
     9→	"bytes"
    10→	"encoding/json"
    11→	"fmt"
    12→	"maps"
    13→	"math"
    14→	"os"
    15→	"path/filepath"
    16→	"strings"
    17→	"text/template"
    18→
    19→	"github.com/ClusterCockpit/cc-backend/internal/repository"
    20→	"github.com/ClusterCockpit/cc-backend/pkg/archive"
    21→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    22→	"github.com/ClusterCockpit/cc-lib/v2/schema"
    23→	"github.com/ClusterCockpit/cc-lib/v2/util"
    24→	"github.com/expr-lang/expr"
    25→	"github.com/expr-lang/expr/vm"
    26→)
    27→
    28→const (
    29→	// defaultJobClassConfigPath is the default path for job classification configuration
    30→	defaultJobClassConfigPath = "./var/tagger/jobclasses"
    31→	// tagTypeJobClass is the tag type identifier for job classification tags
    32→	tagTypeJobClass = "jobClass"
    33→	// jobClassConfigDirMatch is the directory name used for matching filesystem events
    34→	jobClassConfigDirMatch = "jobclasses"
    35→	// parametersFileName is the name of the parameters configuration file
    36→	parametersFileName = "parameters.json"
    37→)
    38→
    39→// Variable defines a named expression that can be computed and reused in rules.
    40→// Variables are evaluated before the main rule and their results are added to the environment.
    41→type Variable struct {
    42→	// Name is the variable identifier used in rule expressions
    43→	Name string `json:"name"`
    44→	// Expr is the expression to evaluate (must return a numeric value)
    45→	Expr string `json:"expr"`
    46→}
    47→
    48→type ruleVariable struct {
    49→	name string
    50→	expr *vm.Program
    51→}
    52→
    53→// RuleFormat defines the JSON structure for job classification rules.
    54→// Each rule specifies requirements, metrics to analyze, variables to compute,
    55→// and the final rule expression that determines if the job matches the classification.
    56→type RuleFormat struct {
    57→	// Name is a human-readable description of the rule
    58→	Name string `json:"name"`
    59→	// Tag is the classification tag to apply if the rule matches
    60→	Tag string `json:"tag"`
    61→	// Parameters are shared values referenced in the rule (e.g., thresholds)
    62→	Parameters []string `json:"parameters"`
    63→	// Metrics are the job metrics required for this rule (e.g., "cpu_load", "mem_used")
    64→	Metrics []string `json:"metrics"`
    65→	// Requirements are boolean expressions that must be true for the rule to apply
    66→	Requirements []string `json:"requirements"`
    67→	// Variables are computed values used in the rule expression
    68→	Variables []Variable `json:"variables"`
    69→	// Rule is the boolean expression that determines if the job matches
    70→	Rule string `json:"rule"`
    71→	// Hint is a template string that generates a message when the rule matches
    72→	Hint string `json:"hint"`
    73→}
    74→
    75→type ruleInfo struct {
    76→	env          map[string]any
    77→	metrics      []string
    78→	requirements []*vm.Program
    79→	variables    []ruleVariable
    80→	rule         *vm.Program
    81→	hint         *template.Template
    82→}
    83→
    84→// JobRepository defines the interface for job database operations needed by the tagger.
    85→// This interface allows for easier testing and decoupling from the concrete repository implementation.
    86→type JobRepository interface {
    87→	// HasTag checks if a job already has a specific tag
    88→	HasTag(jobID int64, tagType string, tagName string) bool
    89→	// AddTagOrCreateDirect adds a tag to a job or creates it if it doesn't exist
    90→	AddTagOrCreateDirect(jobID int64, tagType string, tagName string) (tagID int64, err error)
    91→	// UpdateMetadata updates job metadata with a key-value pair
    92→	UpdateMetadata(job *schema.Job, key, val string) (err error)
    93→}
    94→
    95→// JobClassTagger classifies jobs based on configurable rules that evaluate job metrics and properties.
    96→// Rules are loaded from an external configuration directory and can be dynamically reloaded when files change.
    97→// When a job matches a rule, it is tagged with the corresponding classification and an optional hint message.
    98→type JobClassTagger struct {
    99→	// rules maps classification tags to their compiled rule information
   100→	rules map[string]ruleInfo
   101→	// parameters are shared values (e.g., thresholds) used across multiple rules
   102→	parameters map[string]any
   103→	// tagType is the type of tag ("jobClass")
   104→	tagType string
   105→	// cfgPath is the path to watch for configuration changes
   106→	cfgPath string
   107→	// repo provides access to job database operations
   108→	repo JobRepository
   109→	// getStatistics retrieves job statistics for analysis
   110→	getStatistics func(job *schema.Job) (map[string]schema.JobStatistics, error)
   111→	// getMetricConfig retrieves metric configuration (limits) for a cluster
   112→	getMetricConfig func(cluster, subCluster string) map[string]*schema.Metric
   113→}
   114→
   115→// roundEnv returns a copy of env with all float64 values rounded to 2 decimal places.
   116→// Nested map[string]any and map[string]float64 values are recursed into.
   117→func roundEnv(env map[string]any) map[string]any {
   118→	rounded := make(map[string]any, len(env))
   119→	for k, v := range env {
   120→		switch val := v.(type) {
   121→		case float64:
   122→			rounded[k] = math.Round(val*100) / 100
   123→		case map[string]any:
   124→			rounded[k] = roundEnv(val)
   125→		case map[string]float64:
   126→			rm := make(map[string]float64, len(val))
   127→			for mk, mv := range val {
   128→				rm[mk] = math.Round(mv*100) / 100
   129→			}
   130→			rounded[k] = rm
   131→		default:
   132→			rounded[k] = v
   133→		}
   134→	}
   135→	return rounded
   136→}
   137→
   138→func (t *JobClassTagger) prepareRule(b []byte, fns string) {
   139→	var rule RuleFormat
   140→	if err := json.NewDecoder(bytes.NewReader(b)).Decode(&rule); err != nil {
   141→		cclog.Warn("Error while decoding raw job meta json")
   142→		return
   143→	}
   144→
   145→	ri := ruleInfo{}
   146→	ri.env = make(map[string]any)
   147→	ri.metrics = make([]string, 0)
   148→	ri.requirements = make([]*vm.Program, 0)
   149→	ri.variables = make([]ruleVariable, 0)
   150→
   151→	// check if all required parameters are available
   152→	for _, p := range rule.Parameters {
   153→		param, ok := t.parameters[p]
   154→		if !ok {
   155→			cclog.Warnf("prepareRule() > missing parameter %s in rule %s", p, fns)
   156→			return
   157→		}
   158→		ri.env[p] = param
   159→	}
   160→
   161→	// set all required metrics
   162→	ri.metrics = append(ri.metrics, rule.Metrics...)
   163→
   164→	// compile requirements
   165→	for _, r := range rule.Requirements {
   166→		req, err := expr.Compile(r, expr.AsBool())
   167→		if err != nil {
   168→			cclog.Errorf("error compiling requirement %s: %#v", r, err)
   169→			return
   170→		}
   171→		ri.requirements = append(ri.requirements, req)
   172→	}
   173→
   174→	// compile variables
   175→	for _, v := range rule.Variables {
   176→		req, err := expr.Compile(v.Expr, expr.AsFloat64())
   177→		if err != nil {
   178→			cclog.Errorf("error compiling requirement %s: %#v", v.Name, err)
   179→			return
   180→		}
   181→		ri.variables = append(ri.variables, ruleVariable{name: v.Name, expr: req})
   182→	}
   183→
   184→	// compile rule
   185→	exp, err := expr.Compile(rule.Rule, expr.AsBool())
   186→	if err != nil {
   187→		cclog.Errorf("error compiling rule %s: %#v", fns, err)
   188→		return
   189→	}
   190→	ri.rule = exp
   191→
   192→	// prepare hint template
   193→	ri.hint, err = template.New(fns).Parse(rule.Hint)
   194→	if err != nil {
   195→		cclog.Errorf("error processing template %s: %#v", fns, err)
   196→	}
   197→	cclog.Infof("prepareRule() > processing %s with %d requirements and %d variables", fns, len(ri.requirements), len(ri.variables))
   198→
   199→	t.rules[rule.Tag] = ri
   200→}
   201→
   202→// EventMatch checks if a filesystem event should trigger configuration reload.
   203→// It returns true if the event path contains "jobclasses".
   204→func (t *JobClassTagger) EventMatch(s string) bool {
   205→	return strings.Contains(s, jobClassConfigDirMatch)
   206→}
   207→
   208→// EventCallback is triggered when the configuration directory changes.
   209→// It reloads parameters and all rule files from the watched directory.
   210→// FIXME: Only process the file that caused the event
   211→func (t *JobClassTagger) EventCallback() {
   212→	files, err := os.ReadDir(t.cfgPath)
   213→	if err != nil {
   214→		cclog.Fatal(err)
   215→	}
   216→
   217→	t.rules = make(map[string]ruleInfo)
   218→
   219→	parametersFile := filepath.Join(t.cfgPath, parametersFileName)
   220→	if util.CheckFileExists(parametersFile) {
   221→		cclog.Info("Merge parameters")
   222→		b, err := os.ReadFile(parametersFile)
   223→		if err != nil {
   224→			cclog.Warnf("prepareRule() > open file error: %v", err)
   225→		}
   226→
   227→		var paramTmp map[string]any
   228→		if err := json.NewDecoder(bytes.NewReader(b)).Decode(&paramTmp); err != nil {
   229→			cclog.Warn("Error while decoding parameters.json")
   230→		}
   231→
   232→		maps.Copy(t.parameters, paramTmp)
   233→	}
   234→
   235→	for _, fn := range files {
   236→		fns := fn.Name()
   237→		if fns != parametersFileName {
   238→			cclog.Debugf("Process: %s", fns)
   239→			filename := filepath.Join(t.cfgPath, fns)
   240→			b, err := os.ReadFile(filename)
   241→			if err != nil {
   242→				cclog.Warnf("prepareRule() > open file error: %v", err)
   243→				continue
   244→			}
   245→			t.prepareRule(b, fns)
   246→		}
   247→	}
   248→}
   249→
   250→func (t *JobClassTagger) initParameters() error {
   251→	cclog.Info("Initialize parameters")
   252→	parametersFile := filepath.Join(t.cfgPath, parametersFileName)
   253→	b, err := os.ReadFile(parametersFile)
   254→	if err != nil {
   255→		cclog.Warnf("prepareRule() > open file error: %v", err)
   256→		return err
   257→	}
   258→
   259→	if err := json.NewDecoder(bytes.NewReader(b)).Decode(&t.parameters); err != nil {
   260→		cclog.Warn("Error while decoding parameters.json")
   261→		return err
   262→	}
   263→
   264→	return nil
   265→}
   266→
   267→// Register initializes the JobClassTagger by loading parameters and classification rules from external folder.
   268→// It sets up a file watch on ./var/tagger/jobclasses if it exists, allowing for
   269→// dynamic configuration updates without restarting the application.
   270→// Returns an error if the configuration path does not exist or cannot be read.
   271→func (t *JobClassTagger) Register() error {
   272→	if t.cfgPath == "" {
   273→		t.cfgPath = defaultJobClassConfigPath
   274→	}
   275→	t.tagType = tagTypeJobClass
   276→	t.rules = make(map[string]ruleInfo)
   277→
   278→	if !util.CheckFileExists(t.cfgPath) {
   279→		return fmt.Errorf("configuration path does not exist: %s", t.cfgPath)
   280→	}
   281→
   282→	err := t.initParameters()
   283→	if err != nil {
   284→		cclog.Warnf("error reading parameters.json: %v", err)
   285→		return err
   286→	}
   287→
   288→	files, err := os.ReadDir(t.cfgPath)
   289→	if err != nil {
   290→		return fmt.Errorf("error reading jobclasses folder: %#v", err)
   291→	}
   292→
   293→	for _, fn := range files {
   294→		fns := fn.Name()
   295→		if fns != parametersFileName {
   296→			cclog.Infof("Process: %s", fns)
   297→			filename := filepath.Join(t.cfgPath, fns)
   298→
   299→			b, err := os.ReadFile(filename)
   300→			if err != nil {
   301→				cclog.Warnf("prepareRule() > open file error: %v", err)
   302→				continue
   303→			}
   304→			t.prepareRule(b, fns)
   305→		}
   306→	}
   307→
   308→	cclog.Infof("Setup file watch for %s", t.cfgPath)
   309→	util.AddListener(t.cfgPath, t)
   310→
   311→	t.repo = repository.GetJobRepository()
   312→	t.getStatistics = archive.GetStatistics
   313→	t.getMetricConfig = archive.GetMetricConfigSubCluster
   314→
   315→	return nil
   316→}
   317→
   318→// Match evaluates all classification rules against a job and applies matching tags.
   319→// It retrieves job statistics and metric configurations, then tests each rule's requirements
   320→// and main expression. For each matching rule, it:
   321→//   - Applies the classification tag to the job
   322→//   - Generates and stores a hint message based on the rule's template
   323→//
   324→// The function constructs an evaluation environment containing:
   325→//   - Job properties (duration, cores, nodes, state, etc.)
   326→//   - Metric statistics (min, max, avg) and their configured limits
   327→//   - Shared parameters defined in parameters.json
   328→//   - Computed variables from the rule definition
   329→//
   330→// Rules are evaluated in arbitrary order. Multiple rules can match and apply
   331→// their tags to the same job. Hint messages from all matching rules are collected
   332→// and stored as a combined message in the job metadata.
   333→func (t *JobClassTagger) Match(job *schema.Job) {
   334→	jobStats, err := t.getStatistics(job)
   335→	metricsList := t.getMetricConfig(job.Cluster, job.SubCluster)
   336→	cclog.Debugf("Enter match rule with %d rules for job %d", len(t.rules), job.JobID)
   337→	if err != nil {
   338→		cclog.Errorf("job classification failed for job %d: %#v", job.JobID, err)
   339→		return
   340→	}
   341→
   342→	id := *job.ID
   343→	var messages []string
   344→
   345→	for tag, ri := range t.rules {
   346→		env := make(map[string]any)
   347→		maps.Copy(env, ri.env)
   348→		cclog.Debugf("Try to match rule %s for job %d", tag, job.JobID)
   349→
   350→		// Initialize environment
   351→		env["job"] = map[string]any{
   352→			"shared":   job.Shared,
   353→			"duration": job.Duration,
   354→			"numCores": job.NumHWThreads,
   355→			"numNodes": job.NumNodes,
   356→			"jobState": job.State,
   357→			"numAcc":   job.NumAcc,
   358→			"smt":      job.SMT,
   359→		}
   360→
   361→		// add metrics to env
   362→		skipRule := false
   363→		for _, m := range ri.metrics {
   364→			stats, ok := jobStats[m]
   365→			if !ok {
   366→				cclog.Errorf("job classification: missing metric '%s' for rule %s on job %d", m, tag, job.JobID)
   367→				skipRule = true
   368→				break
   369→			}
   370→			env[m] = map[string]any{
   371→				"min": stats.Min,
   372→				"max": stats.Max,
   373→				"avg": stats.Avg,
   374→				"limits": map[string]float64{
   375→					"peak":    metricsList[m].Peak,
   376→					"normal":  metricsList[m].Normal,
   377→					"caution": metricsList[m].Caution,
   378→					"alert":   metricsList[m].Alert,
   379→				},
   380→			}
   381→		}
   382→		if skipRule {
   383→			continue
   384→		}
   385→
   386→		// check rule requirements apply
   387→		requirementsMet := true
   388→		for _, r := range ri.requirements {
   389→			ok, err := expr.Run(r, env)
   390→			if err != nil {
   391→				cclog.Errorf("error running requirement for rule %s: %#v", tag, err)
   392→				requirementsMet = false
   393→				break
   394→			}
   395→			if !ok.(bool) {
   396→				cclog.Debugf("requirement for rule %s not met", tag)
   397→				requirementsMet = false
   398→				break
   399→			}
   400→		}
   401→		if !requirementsMet {
   402→			continue
   403→		}
   404→
   405→		// evaluate rule variables
   406→		varError := false
   407→		for _, v := range ri.variables {
   408→			value, err := expr.Run(v.expr, env)
   409→			if err != nil {
   410→				cclog.Errorf("error evaluating variable %s for rule %s: %#v", v.name, tag, err)
   411→				varError = true
   412→				break
   413→			}
   414→			env[v.name] = value
   415→		}
   416→		if varError {
   417→			continue
   418→		}
   419→
   420→		match, err := expr.Run(ri.rule, env)
   421→		if err != nil {
   422→			cclog.Errorf("error running rule %s: %#v", tag, err)
   423→			continue
   424→		}
   425→		if match.(bool) {
   426→			if !t.repo.HasTag(id, t.tagType, tag) {
   427→				if _, err := t.repo.AddTagOrCreateDirect(id, t.tagType, tag); err != nil {
   428→					cclog.Errorf("failed to add tag '%s' to job %d: %v", tag, id, err)
   429→					continue
   430→				}
   431→			}
   432→
   433→			// process hint template
   434→			var msg bytes.Buffer
   435→			if err := ri.hint.Execute(&msg, roundEnv(env)); err != nil {
   436→				cclog.Errorf("Template error: %s", err.Error())
   437→				continue
   438→			}
   439→			messages = append(messages, msg.String())
   440→		}
   441→	}
   442→
   443→	if len(messages) > 0 {
   444→		combined := strings.Join(messages, "\n")
   445→		if err := t.repo.UpdateMetadata(job, "message", combined); err != nil {
   446→			cclog.Errorf("failed to update metadata for job %d: %v", *job.ID, err)
   447→		}
   448→	}
   449→}
   450→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package tagger

import (
	"bytes"
	"encoding/json"
	"fmt"
	"maps"
	"math"
	"os"
	"path/filepath"
	"strings"
	"text/template"

	"github.com/ClusterCockpit/cc-backend/internal/repository"
	"github.com/ClusterCockpit/cc-backend/pkg/archive"
	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/schema"
	"github.com/ClusterCockpit/cc-lib/v2/util"
	"github.com/expr-lang/expr"
	"github.com/expr-lang/expr/vm"
)

const (
	// defaultJobClassConfigPath is the default path for job classification configuration
	defaultJobClassConfigPath = "./var/tagger/jobclasses"
	// tagTypeJobClass is the tag type identifier for job classification tags
	tagTypeJobClass = "jobClass"
	// jobClassConfigDirMatch is the directory name used for matching filesystem events
	jobClassConfigDirMatch = "jobclasses"
	// parametersFileName is the name of the parameters configuration file
	parametersFileName = "parameters.json"
)

// Variable defines a named expression that can be computed and reused in rules.
// Variables are evaluated before the main rule and their results are added to the environment.
type Variable struct {
	// Name is the variable identifier used in rule expressions
	Name string `json:"name"`
	// Expr is the expression to evaluate (must return a numeric value)
	Expr string `json:"expr"`
}

type ruleVariable struct {
	name string
	expr *vm.Program
}

// RuleFormat defines the JSON structure for job classification rules.
// Each rule specifies requirements, metrics to analyze, variables to compute,
// and the final rule expression that determines if the job matches the classification.
type RuleFormat struct {
	// Name is a human-readable description of the rule
	Name string `json:"name"`
	// Tag is the classification tag to apply if the rule matches
	Tag string `json:"tag"`
	// Parameters are shared values referenced in the rule (e.g., thresholds)
	Parameters []string `json:"parameters"`
	// Metrics are the job metrics required for this rule (e.g., "cpu_load", "mem_used")
	Metrics []string `json:"metrics"`
	// Requirements are boolean expressions that must be true for the rule to apply
	Requirements []string `json:"requirements"`
	// Variables are computed values used in the rule expression
	Variables []Variable `json:"variables"`
	// Rule is the boolean expression that determines if the job matches
	Rule string `json:"rule"`
	// Hint is a template string that generates a message when the rule matches
	Hint string `json:"hint"`
}

type ruleInfo struct {
	env          map[string]any
	metrics      []string
	requirements []*vm.Program
	variables    []ruleVariable
	rule         *vm.Program
	hint         *template.Template
}

// JobRepository defines the interface for job database operations needed by the tagger.
// This interface allows for easier testing and decoupling from the concrete repository implementation.
type JobRepository interface {
	// HasTag checks if a job already has a specific tag
	HasTag(jobID int64, tagType string, tagName string) bool
	// AddTagOrCreateDirect adds a tag to a job or creates it if it doesn't exist
	AddTagOrCreateDirect(jobID int64, tagType string, tagName string) (tagID int64, err error)
	// UpdateMetadata updates job metadata with a key-value pair
	UpdateMetadata(job *schema.Job, key, val string) (err error)
}

// JobClassTagger classifies jobs based on configurable rules that evaluate job metrics and properties.
// Rules are loaded from an external configuration directory and can be dynamically reloaded when files change.
// When a job matches a rule, it is tagged with the corresponding classification and an optional hint message.
type JobClassTagger struct {
	// rules maps classification tags to their compiled rule information
	rules map[string]ruleInfo
	// parameters are shared values (e.g., thresholds) used across multiple rules
	parameters map[string]any
	// tagType is the type of tag ("jobClass")
	tagType string
	// cfgPath is the path to watch for configuration changes
	cfgPath string
	// repo provides access to job database operations
	repo JobRepository
	// getStatistics retrieves job statistics for analysis
	getStatistics func(job *schema.Job) (map[string]schema.JobStatistics, error)
	// getMetricConfig retrieves metric configuration (limits) for a cluster
	getMetricConfig func(cluster, subCluster string) map[string]*schema.Metric
}

// roundEnv returns a copy of env with all float64 values rounded to 2 decimal places.
// Nested map[string]any and map[string]float64 values are recursed into.
func roundEnv(env map[string]any) map[string]any {
	rounded := make(map[string]any, len(env))
	for k, v := range env {
		switch val := v.(type) {
		case float64:
			rounded[k] = math.Round(val*100) / 100
		case map[string]any:
			rounded[k] = roundEnv(val)
		case map[string]float64:
			rm := make(map[string]float64, len(val))
			for mk, mv := range val {
				rm[mk] = math.Round(mv*100) / 100
			}
			rounded[k] = rm
		default:
			rounded[k] = v
		}
	}
	return rounded
}

func (t *JobClassTagger) prepareRule(b []byte, fns string) {
	var rule RuleFormat
	if err := json.NewDecoder(bytes.NewReader(b)).Decode(&rule); err != nil {
		cclog.Warn("Error while decoding raw job meta json")
		return
	}

	ri := ruleInfo{}
	ri.env = make(map[string]any)
	ri.metrics = make([]string, 0)
	ri.requirements = make([]*vm.Program, 0)
	ri.variables = make([]ruleVariable, 0)

	// check if all required parameters are available
	for _, p := range rule.Parameters {
		param, ok := t.parameters[p]
		if !ok {
			cclog.Warnf("prepareRule() > missing parameter %s in rule %s", p, fns)
			return
		}
		ri.env[p] = param
	}

	// set all required metrics
	ri.metrics = append(ri.metrics, rule.Metrics...)

	// compile requirements
	for _, r := range rule.Requirements {
		req, err := expr.Compile(r, expr.AsBool())
		if err != nil {
			cclog.Errorf("error compiling requirement %s: %#v", r, err)
			return
		}
		ri.requirements = append(ri.requirements, req)
	}

	// compile variables
	for _, v := range rule.Variables {
		req, err := expr.Compile(v.Expr, expr.AsFloat64())
		if err != nil {
			cclog.Errorf("error compiling requirement %s: %#v", v.Name, err)
			return
		}
		ri.variables = append(ri.variables, ruleVariable{name: v.Name, expr: req})
	}

	// compile rule
	exp, err := expr.Compile(rule.Rule, expr.AsBool())
	if err != nil {
		cclog.Errorf("error compiling rule %s: %#v", fns, err)
		return
	}
	ri.rule = exp

	// prepare hint template
	ri.hint, err = template.New(fns).Parse(rule.Hint)
	if err != nil {
		cclog.Errorf("error processing template %s: %#v", fns, err)
	}
	cclog.Infof("prepareRule() > processing %s with %d requirements and %d variables", fns, len(ri.requirements), len(ri.variables))

	t.rules[rule.Tag] = ri
}

// EventMatch checks if a filesystem event should trigger configuration reload.
// It returns true if the event path contains "jobclasses".
func (t *JobClassTagger) EventMatch(s string) bool {
	return strings.Contains(s, jobClassConfigDirMatch)
}

// EventCallback is triggered when the configuration directory changes.
// It reloads parameters and all rule files from the watched directory.
// FIXME: Only process the file that caused the event
func (t *JobClassTagger) EventCallback() {
	files, err := os.ReadDir(t.cfgPath)
	if err != nil {
		cclog.Fatal(err)
	}

	t.rules = make(map[string]ruleInfo)

	parametersFile := filepath.Join(t.cfgPath, parametersFileName)
	if util.CheckFileExists(parametersFile) {
		cclog.Info("Merge parameters")
		b, err := os.ReadFile(parametersFile)
		if err != nil {
			cclog.Warnf("prepareRule() > open file error: %v", err)
		}

		var paramTmp map[string]any
		if err := json.NewDecoder(bytes.NewReader(b)).Decode(&paramTmp); err != nil {
			cclog.Warn("Error while decoding parameters.json")
		}

		maps.Copy(t.parameters, paramTmp)
	}

	for _, fn := range files {
		fns := fn.Name()
		if fns != parametersFileName {
			cclog.Debugf("Process: %s", fns)
			filename := filepath.Join(t.cfgPath, fns)
			b, err := os.ReadFile(filename)
			if err != nil {
				cclog.Warnf("prepareRule() > open file error: %v", err)
				continue
			}
			t.prepareRule(b, fns)
		}
	}
}

func (t *JobClassTagger) initParameters() error {
	cclog.Info("Initialize parameters")
	parametersFile := filepath.Join(t.cfgPath, parametersFileName)
	b, err := os.ReadFile(parametersFile)
	if err != nil {
		cclog.Warnf("prepareRule() > open file error: %v", err)
		return err
	}

	if err := json.NewDecoder(bytes.NewReader(b)).Decode(&t.parameters); err != nil {
		cclog.Warn("Error while decoding parameters.json")
		return err
	}

	return nil
}

// Register initializes the JobClassTagger by loading parameters and classification rules from external folder.
// It sets up a file watch on ./var/tagger/jobclasses if it exists, allowing for
// dynamic configuration updates without restarting the application.
// Returns an error if the configuration path does not exist or cannot be read.
func (t *JobClassTagger) Register() error {
	if t.cfgPath == "" {
		t.cfgPath = defaultJobClassConfigPath
	}
	t.tagType = tagTypeJobClass
	t.rules = make(map[string]ruleInfo)

	if !util.CheckFileExists(t.cfgPath) {
		return fmt.Errorf("configuration path does not exist: %s", t.cfgPath)
	}

	err := t.initParameters()
	if err != nil {
		cclog.Warnf("error reading parameters.json: %v", err)
		return err
	}

	files, err := os.ReadDir(t.cfgPath)
	if err != nil {
		return fmt.Errorf("error reading jobclasses folder: %#v", err)
	}

	for _, fn := range files {
		fns := fn.Name()
		if fns != parametersFileName {
			cclog.Infof("Process: %s", fns)
			filename := filepath.Join(t.cfgPath, fns)

			b, err := os.ReadFile(filename)
			if err != nil {
				cclog.Warnf("prepareRule() > open file error: %v", err)
				continue
			}
			t.prepareRule(b, fns)
		}
	}

	cclog.Infof("Setup file watch for %s", t.cfgPath)
	util.AddListener(t.cfgPath, t)

	t.repo = repository.GetJobRepository()
	t.getStatistics = archive.GetStatistics
	t.getMetricConfig = archive.GetMetricConfigSubCluster

	return nil
}

// Match evaluates all classification rules against a job and applies matching tags.
// It retrieves job statistics and metric configurations, then tests each rule's requirements
// and main expression. For each matching rule, it:
//   - Applies the classification tag to the job
//   - Generates and stores a hint message based on the rule's template
//
// The function constructs an evaluation environment containing:
//   - Job properties (duration, cores, nodes, state, etc.)
//   - Metric statistics (min, max, avg) and their configured limits
//   - Shared parameters defined in parameters.json
//   - Computed variables from the rule definition
//
// Rules are evaluated in arbitrary order. Multiple rules can match and apply
// their tags to the same job. Hint messages from all matching rules are collected
// and stored as a combined message in the job metadata.
func (t *JobClassTagger) Match(job *schema.Job) {
	jobStats, err := t.getStatistics(job)
	metricsList := t.getMetricConfig(job.Cluster, job.SubCluster)
	cclog.Debugf("Enter match rule with %d rules for job %d", len(t.rules), job.JobID)
	if err != nil {
		cclog.Errorf("job classification failed for job %d: %#v", job.JobID, err)
		return
	}

	id := *job.ID
	var messages []string

	for tag, ri := range t.rules {
		env := make(map[string]any)
		maps.Copy(env, ri.env)
		cclog.Debugf("Try to match rule %s for job %d", tag, job.JobID)

		// Initialize environment
		env["job"] = map[string]any{
			"shared":   job.Shared,
			"duration": job.Duration,
			"numCores": job.NumHWThreads,
			"numNodes": job.NumNodes,
			"jobState": job.State,
			"numAcc":   job.NumAcc,
			"smt":      job.SMT,
		}

		// add metrics to env
		skipRule := false
		for _, m := range ri.metrics {
			stats, ok := jobStats[m]
			if !ok {
				cclog.Errorf("job classification: missing metric '%s' for rule %s on job %d", m, tag, job.JobID)
				skipRule = true
				break
			}
			env[m] = map[string]any{
				"min": stats.Min,
				"max": stats.Max,
				"avg": stats.Avg,
				"limits": map[string]float64{
					"peak":    metricsList[m].Peak,
					"normal":  metricsList[m].Normal,
					"caution": metricsList[m].Caution,
					"alert":   metricsList[m].Alert,
				},
			}
		}
		if skipRule {
			continue
		}

		// check rule requirements apply
		requirementsMet := true
		for _, r := range ri.requirements {
			ok, err := expr.Run(r, env)
			if err != nil {
				cclog.Errorf("error running requirement for rule %s: %#v", tag, err)
				requirementsMet = false
				break
			}
			if !ok.(bool) {
				cclog.Debugf("requirement for rule %s not met", tag)
				requirementsMet = false
				break
			}
		}
		if !requirementsMet {
			continue
		}

		// evaluate rule variables
		varError := false
		for _, v := range ri.variables {
			value, err := expr.Run(v.expr, env)
			if err != nil {
				cclog.Errorf("error evaluating variable %s for rule %s: %#v", v.name, tag, err)
				varError = true
				break
			}
			env[v.name] = value
		}
		if varError {
			continue
		}

		match, err := expr.Run(ri.rule, env)
		if err != nil {
			cclog.Errorf("error running rule %s: %#v", tag, err)
			continue
		}
		if match.(bool) {
			if !t.repo.HasTag(id, t.tagType, tag) {
				if _, err := t.repo.AddTagOrCreateDirect(id, t.tagType, tag); err != nil {
					cclog.Errorf("failed to add tag '%s' to job %d: %v", tag, id, err)
					continue
				}
			}

			// process hint template
			var msg bytes.Buffer
			if err := ri.hint.Execute(&msg, roundEnv(env)); err != nil {
				cclog.Errorf("Template error: %s", err.Error())
				continue
			}
			messages = append(messages, msg.String())
		}
	}

	if len(messages) > 0 {
		combined := strings.Join(messages, "\n")
		if err := t.repo.UpdateMetadata(job, "message", combined); err != nil {
			cclog.Errorf("failed to update metadata for job %d: %v", *job.ID, err)
		}
	}
}


Found the bug. In `newTagger()` (tagger.go:67-84), taggers are appended to the slices **before** `Register()` is called. When `Register()` fails (directory missing), the tagger stays in the slice with uninitialized fields (`t.repo`, `t.getStatistics`, `t.getMetricConfig` are all `nil`). When `Match()` is subsequently called, it dereferences these nil function pointers at `classifyJob.go:334-335`, causing the segfault.

The fix is to register each tagger first, and only add it to the slice if registration succeeds:


Edit

func newTagger() {
	jobTagger = &JobTagger{}
	jobTagger.startTaggers = make([]Tagger, 0)
	jobTagger.startTaggers = append(jobTagger.startTaggers, &AppTagger{})
	jobTagger.stopTaggers = make([]Tagger, 0)
	jobTagger.stopTaggers = append(jobTagger.stopTaggers, &JobClassTagger{})

	for _, tagger := range jobTagger.startTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register start tagger: %s", err)
		}
	}
	for _, tagger := range jobTagger.stopTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register stop tagger: %s", err)
		}
	}
}

func newTagger() {
	jobTagger = &JobTagger{}
	jobTagger.startTaggers = make([]Tagger, 0)
	jobTagger.stopTaggers = make([]Tagger, 0)

	for _, t := range []Tagger{&AppTagger{}} {
		if err := t.Register(); err != nil {
			cclog.Errorf("failed to register start tagger: %s", err)
		} else {
			jobTagger.startTaggers = append(jobTagger.startTaggers, t)
		}
	}
	for _, t := range []Tagger{&JobClassTagger{}} {
		if err := t.Register(); err != nil {
			cclog.Errorf("failed to register stop tagger: %s", err)
		} else {
			jobTagger.stopTaggers = append(jobTagger.stopTaggers, t)
		}
	}
}


The file /Users/jan/prg/ClusterCockpit/cc-backend/internal/tagger/tagger.go has been updated successfully.

func newTagger() {
	jobTagger = &JobTagger{}
	jobTagger.startTaggers = make([]Tagger, 0)
	jobTagger.stopTaggers = make([]Tagger, 0)

	for _, t := range []Tagger{&AppTagger{}} {
		if err := t.Register(); err != nil {
			cclog.Errorf("failed to register start tagger: %s", err)
		} else {
			jobTagger.startTaggers = append(jobTagger.startTaggers, t)
		}
	}
	for _, t := range []Tagger{&JobClassTagger{}} {
		if err := t.Register(); err != nil {
			cclog.Errorf("failed to register stop tagger: %s", err)
		} else {
			jobTagger.stopTaggers = append(jobTagger.stopTaggers, t)
		}
	}
}

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

// Package tagger provides automatic job tagging functionality for cc-backend.
// It supports detecting applications and classifying jobs based on configurable rules.
// Tags are automatically applied when jobs start or stop, or can be applied retroactively
// to existing jobs using RunTaggers.
package tagger

import (
	"fmt"
	"sync"

	"github.com/ClusterCockpit/cc-backend/internal/repository"
	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/ClusterCockpit/cc-lib/v2/schema"
)

// Tagger is the interface that must be implemented by all tagging components.
// Taggers can be registered at job start or stop events to automatically apply tags.
type Tagger interface {
	// Register initializes the tagger and loads any required configuration.
	// It should be called once before the tagger is used.
	Register() error

	// Match evaluates the tagger's rules against a job and applies appropriate tags.
	// It is called for each job that needs to be evaluated.
	Match(job *schema.Job)
}

// TaggerInfo holds metadata about a tagger for JSON serialization.
type TaggerInfo struct {
	Name    string `json:"name"`
	Type    string `json:"type"`
	Running bool   `json:"running"`
}

var (
	initOnce     sync.Once
	jobTagger    *JobTagger
	statusMu     sync.Mutex
	taggerStatus = map[string]bool{}
)

// Known tagger definitions: name -> (type, factory)
type taggerDef struct {
	ttype   string
	factory func() Tagger
}

var knownTaggers = map[string]taggerDef{
	"AppTagger":      {ttype: "start", factory: func() Tagger { return &AppTagger{} }},
	"JobClassTagger": {ttype: "stop", factory: func() Tagger { return &JobClassTagger{} }},
}

// JobTagger coordinates multiple taggers that run at different job lifecycle events.
// It maintains separate lists of taggers that run when jobs start and when they stop.
type JobTagger struct {
	// startTaggers are applied when a job starts (e.g., application detection)
	startTaggers []Tagger
	// stopTaggers are applied when a job completes (e.g., job classification)
	stopTaggers []Tagger
}

func newTagger() {
	jobTagger = &JobTagger{}
	jobTagger.startTaggers = make([]Tagger, 0)
	jobTagger.startTaggers = append(jobTagger.startTaggers, &AppTagger{})
	jobTagger.stopTaggers = make([]Tagger, 0)
	jobTagger.stopTaggers = append(jobTagger.stopTaggers, &JobClassTagger{})

	for _, tagger := range jobTagger.startTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register start tagger: %s", err)
		}
	}
	for _, tagger := range jobTagger.stopTaggers {
		if err := tagger.Register(); err != nil {
			cclog.Errorf("failed to register stop tagger: %s", err)
		}
	}
}

// Init initializes the job tagger system and registers it with the job repository.
// This function is safe to call multiple times; initialization only occurs once.
// It should be called during application startup.
func Init() {
	initOnce.Do(func() {
		newTagger()
		repository.RegisterJobHook(jobTagger)
	})
}

// JobStartCallback is called when a job starts.
// It runs all registered start taggers (e.g., application detection) on the job.
func (jt *JobTagger) JobStartCallback(job *schema.Job) {
	for _, tagger := range jt.startTaggers {
		tagger.Match(job)
	}
}

// JobStopCallback is called when a job completes.
// It runs all registered stop taggers (e.g., job classification) on the job.
func (jt *JobTagger) JobStopCallback(job *schema.Job) {
	for _, tagger := range jt.stopTaggers {
		tagger.Match(job)
	}
}

// ListTaggers returns information about all known taggers with their current running status.
func ListTaggers() []TaggerInfo {
	statusMu.Lock()
	defer statusMu.Unlock()

	result := make([]TaggerInfo, 0, len(knownTaggers))
	for name, def := range knownTaggers {
		result = append(result, TaggerInfo{
			Name:    name,
			Type:    def.ttype,
			Running: taggerStatus[name],
		})
	}
	return result
}

// RunTaggerByName starts a tagger by name asynchronously on all jobs.
// Returns an error if the name is unknown or the tagger is already running.
func RunTaggerByName(name string) error {
	def, ok := knownTaggers[name]
	if !ok {
		return fmt.Errorf("unknown tagger: %s", name)
	}

	statusMu.Lock()
	if taggerStatus[name] {
		statusMu.Unlock()
		return fmt.Errorf("tagger %s is already running", name)
	}
	taggerStatus[name] = true
	statusMu.Unlock()

	go func() {
		defer func() {
			statusMu.Lock()
			taggerStatus[name] = false
			statusMu.Unlock()
		}()

		t := def.factory()
		if err := t.Register(); err != nil {
			cclog.Errorf("Failed to register tagger %s: %s", name, err)
			return
		}

		r := repository.GetJobRepository()
		jl, err := r.GetJobList(0, 0)
		if err != nil {
			cclog.Errorf("Error getting job list for tagger %s: %s", name, err)
			return
		}

		cclog.Infof("Running tagger %s on %d jobs", name, len(jl))
		for _, id := range jl {
			job, err := r.FindByIDDirect(id)
			if err != nil {
				cclog.Errorf("Error getting job %d for tagger %s: %s", id, name, err)
				continue
			}
			t.Match(job)
		}
		cclog.Infof("Tagger %s completed", name)
	}()

	return nil
}

// RunTaggers applies all configured taggers to all existing jobs in the repository.
// This is useful for retroactively applying tags to jobs that were created before
// the tagger system was initialized or when new tagging rules are added.
// It fetches all jobs and runs both start and stop taggers on each one.
func RunTaggers() error {
	newTagger()
	r := repository.GetJobRepository()
	jl, err := r.GetJobList(0, 0) // 0 limit means get all jobs (no pagination)
	if err != nil {
		cclog.Errorf("Error while getting job list %s", err)
		return err
	}

	for _, id := range jl {
		job, err := r.FindByIDDirect(id)
		if err != nil {
			cclog.Errorf("Error while getting job %s", err)
			return err
		}
		for _, tagger := range jobTagger.startTaggers {
			tagger.Match(job)
		}
		for _, tagger := range jobTagger.stopTaggers {
			cclog.Infof("Run stop tagger for job %d", *job.ID)
			tagger.Match(job)
		}
	}
	return nil
}

-  jobTagger.startTaggers = append(jobTagger.startTaggers, &AppTagger{})

-  jobTagger.stopTaggers = append(jobTagger.stopTaggers, &JobClassTagger{})

-  for _, tagger := range jobTagger.startTaggers {

-    if err := tagger.Register(); err != nil {

+  for _, t := range []Tagger{&AppTagger{}} {

+    if err := t.Register(); err != nil {

+      jobTagger.startTaggers = append(jobTagger.startTaggers, t)

-  for _, tagger := range jobTagger.stopTaggers {

+  for _, t := range []Tagger{&JobClassTagger{}} {

+      jobTagger.stopTaggers = append(jobTagger.stopTaggers, t)


Now verify the fix compiles:


Bash

Build tagger package


Builds cleanly.

**Root cause:** Taggers were appended to the slices unconditionally, then `Register()` was called. A failed registration (missing directory) left `JobClassTagger.repo`, `.getStatistics`, and `.getMetricConfig` as `nil`. The next `Match()` call dereferenced those nil function pointers → segfault.

**Fix:** Register each tagger first; only append it to the active slice on success. Failed taggers are logged and skipped.