   130→./cc-backend -server  -dev
   131→
   132→# Show other options:
   133→./cc-backend -help
   134→```
   135→
   136→## Project file structure
   137→
   138→- [`.github/`](https://github.com/ClusterCockpit/cc-backend/tree/master/.github)
   139→  GitHub Actions workflows and dependabot configuration for CI/CD.
   140→- [`api/`](https://github.com/ClusterCockpit/cc-backend/tree/master/api)
   141→  contains the API schema files for the REST and GraphQL APIs. The REST API is
   142→  documented in the OpenAPI 3.0 format in
   143→  [./api/swagger.yaml](./api/swagger.yaml). The GraphQL schema is in
   144→  [./api/schema.graphqls](./api/schema.graphqls).
   145→- [`cmd/cc-backend`](https://github.com/ClusterCockpit/cc-backend/tree/master/cmd/cc-backend)
   146→  contains the main application entry point and CLI implementation.
   147→- [`configs/`](https://github.com/ClusterCockpit/cc-backend/tree/master/configs)
   148→  contains documentation about configuration and command line options and required
   149→  environment variables. Sample configuration files are provided.
   150→- [`init/`](https://github.com/ClusterCockpit/cc-backend/tree/master/init)
   151→  contains an example of setting up systemd for production use.
   152→- [`internal/`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal)
   153→  contains library source code that is not intended for use by others.
   154→  - [`api`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/api)
   155→    REST API handlers and NATS integration
   156→  - [`archiver`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/archiver)
   157→    Job archiving functionality
   158→  - [`auth`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/auth)
   159→    Authentication (local, LDAP, OIDC) and JWT token handling
   160→  - [`config`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/config)
   161→    Configuration management and validation
   162→  - [`graph`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/graph)
   163→    GraphQL schema and resolvers
   164→  - [`importer`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/importer)
   165→    Job data import and database initialization
   166→  - [`metricdispatch`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricdispatch)
   167→    Dispatches metric data loading to appropriate backends
   168→  - [`repository`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/repository)
   169→    Database repository layer for jobs and metadata
   170→  - [`routerConfig`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/routerConfig)
   171→    HTTP router configuration and middleware
   172→  - [`tagger`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/tagger)
   173→    Job classification and application detection
   174→  - [`taskmanager`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/taskmanager)
   175→    Background task management and scheduled jobs
   176→  - [`metricstoreclient`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricstoreclient)
   177→    Client for cc-metric-store queries
   178→- [`pkg/`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg)
   179→  contains Go packages that can be used by other projects.
   180→  - [`archive`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/archive)
   181→    Job archive backend implementations (filesystem, S3, SQLite)
   182→  - [`metricstore`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/metricstore)
   183→    In-memory metric data store with checkpointing and metric loading
   184→- [`tools/`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools)
   185→  Additional command line helper tools.
   186→  - [`archive-manager`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-manager)
   187→    Commands for getting infos about an existing job archive, importing jobs
   188→    between archive backends, and converting archives between JSON and Parquet formats.
   189→  - [`archive-migration`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-migration)
   190→    Tool for migrating job archives between formats.
   191→  - [`convert-pem-pubkey`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/convert-pem-pubkey)
   192→    Tool to convert external pubkey for use in `cc-backend`.
   193→  - [`gen-keypair`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/gen-keypair)
   194→    contains a small application to generate a compatible JWT keypair. You find
   195→    documentation on how to use it
   196→    [here](https://github.com/ClusterCockpit/cc-backend/blob/master/docs/JWT-Handling.md).
   197→- [`web/`](https://github.com/ClusterCockpit/cc-backend/tree/master/web)
   198→  Server-side templates and frontend-related files:
   199→  - [`frontend`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/frontend)
   200→    Svelte components and static assets for the frontend UI
   201→  - [`templates`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/templates)
   202→    Server-side Go templates, including monitoring views
   203→- [`gqlgen.yml`](https://github.com/ClusterCockpit/cc-backend/blob/master/gqlgen.yml)
   204→  Configures the behaviour and generation of
   205→  [gqlgen](https://github.com/99designs/gqlgen).
   206→- [`startDemo.sh`](https://github.com/ClusterCockpit/cc-backend/blob/master/startDemo.sh)
   207→  is a shell script that sets up demo data, and builds and starts `cc-backend`.
   208→

./cc-backend -server  -dev

# Show other options:
./cc-backend -help
```

## Project file structure

- [`.github/`](https://github.com/ClusterCockpit/cc-backend/tree/master/.github)
  GitHub Actions workflows and dependabot configuration for CI/CD.
- [`api/`](https://github.com/ClusterCockpit/cc-backend/tree/master/api)
  contains the API schema files for the REST and GraphQL APIs. The REST API is
  documented in the OpenAPI 3.0 format in
  [./api/swagger.yaml](./api/swagger.yaml). The GraphQL schema is in
  [./api/schema.graphqls](./api/schema.graphqls).
- [`cmd/cc-backend`](https://github.com/ClusterCockpit/cc-backend/tree/master/cmd/cc-backend)
  contains the main application entry point and CLI implementation.
- [`configs/`](https://github.com/ClusterCockpit/cc-backend/tree/master/configs)
  contains documentation about configuration and command line options and required
  environment variables. Sample configuration files are provided.
- [`init/`](https://github.com/ClusterCockpit/cc-backend/tree/master/init)
  contains an example of setting up systemd for production use.
- [`internal/`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal)
  contains library source code that is not intended for use by others.
  - [`api`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/api)
    REST API handlers and NATS integration
  - [`archiver`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/archiver)
    Job archiving functionality
  - [`auth`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/auth)
    Authentication (local, LDAP, OIDC) and JWT token handling
  - [`config`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/config)
    Configuration management and validation
  - [`graph`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/graph)
    GraphQL schema and resolvers
  - [`importer`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/importer)
    Job data import and database initialization
  - [`metricdispatch`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricdispatch)
    Dispatches metric data loading to appropriate backends
  - [`repository`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/repository)
    Database repository layer for jobs and metadata
  - [`routerConfig`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/routerConfig)
    HTTP router configuration and middleware
  - [`tagger`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/tagger)
    Job classification and application detection
  - [`taskmanager`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/taskmanager)
    Background task management and scheduled jobs
  - [`metricstoreclient`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricstoreclient)
    Client for cc-metric-store queries
- [`pkg/`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg)
  contains Go packages that can be used by other projects.
  - [`archive`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/archive)
    Job archive backend implementations (filesystem, S3, SQLite)
  - [`metricstore`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/metricstore)
    In-memory metric data store with checkpointing and metric loading
- [`tools/`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools)
  Additional command line helper tools.
  - [`archive-manager`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-manager)
    Commands for getting infos about an existing job archive, importing jobs
    between archive backends, and converting archives between JSON and Parquet formats.
  - [`archive-migration`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-migration)
    Tool for migrating job archives between formats.
  - [`convert-pem-pubkey`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/convert-pem-pubkey)
    Tool to convert external pubkey for use in `cc-backend`.
  - [`gen-keypair`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/gen-keypair)
    contains a small application to generate a compatible JWT keypair. You find
    documentation on how to use it
    [here](https://github.com/ClusterCockpit/cc-backend/blob/master/docs/JWT-Handling.md).
- [`web/`](https://github.com/ClusterCockpit/cc-backend/tree/master/web)
  Server-side templates and frontend-related files:
  - [`frontend`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/frontend)
    Svelte components and static assets for the frontend UI
  - [`templates`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/templates)
    Server-side Go templates, including monitoring views
- [`gqlgen.yml`](https://github.com/ClusterCockpit/cc-backend/blob/master/gqlgen.yml)
  Configures the behaviour and generation of
  [gqlgen](https://github.com/99designs/gqlgen).
- [`startDemo.sh`](https://github.com/ClusterCockpit/cc-backend/blob/master/startDemo.sh)
  is a shell script that sets up demo data, and builds and starts `cc-backend`.


Now I'll also check the current state of the code files to see what the user changed:


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package repository
     7→
     8→import "time"
     9→
    10→// RepositoryConfig holds configuration for repository operations.
    11→// All fields have sensible defaults, so this configuration is optional.
    12→type RepositoryConfig struct {
    13→	// CacheSize is the LRU cache size in bytes for job metadata and energy footprints.
    14→	// Default: 1MB (1024 * 1024 bytes)
    15→	CacheSize int
    16→
    17→	// MaxOpenConnections is the maximum number of open database connections.
    18→	// Default: 4
    19→	MaxOpenConnections int
    20→
    21→	// MaxIdleConnections is the maximum number of idle database connections.
    22→	// Default: 4
    23→	MaxIdleConnections int
    24→
    25→	// ConnectionMaxLifetime is the maximum amount of time a connection may be reused.
    26→	// Default: 1 hour
    27→	ConnectionMaxLifetime time.Duration
    28→
    29→	// ConnectionMaxIdleTime is the maximum amount of time a connection may be idle.
    30→	// Default: 1 hour
    31→	ConnectionMaxIdleTime time.Duration
    32→
    33→	// MinRunningJobDuration is the minimum duration in seconds for a job to be
    34→	// considered in "running jobs" queries. This filters out very short jobs.
    35→	// Default: 600 seconds (10 minutes)
    36→	MinRunningJobDuration int
    37→}
    38→
    39→// DefaultConfig returns the default repository configuration.
    40→// These values are optimized for typical deployments.
    41→func DefaultConfig() *RepositoryConfig {
    42→	return &RepositoryConfig{
    43→		CacheSize:             1 * 1024 * 1024, // 1MB
    44→		MaxOpenConnections:    4,
    45→		MaxIdleConnections:    4,
    46→		ConnectionMaxLifetime: time.Hour,
    47→		ConnectionMaxIdleTime: time.Hour,
    48→		MinRunningJobDuration: 600, // 10 minutes
    49→	}
    50→}
    51→
    52→// repoConfig is the package-level configuration instance.
    53→// It is initialized with defaults and can be overridden via SetConfig.
    54→var repoConfig *RepositoryConfig = DefaultConfig()
    55→
    56→// SetConfig sets the repository configuration.
    57→// This must be called before any repository initialization (Connect, GetJobRepository, etc.).
    58→// If not called, default values from DefaultConfig() are used.
    59→func SetConfig(cfg *RepositoryConfig) {
    60→	if cfg != nil {
    61→		repoConfig = cfg
    62→	}
    63→}
    64→
    65→// GetConfig returns the current repository configuration.
    66→func GetConfig() *RepositoryConfig {
    67→	return repoConfig
    68→}
    69→

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package repository

import "time"

// RepositoryConfig holds configuration for repository operations.
// All fields have sensible defaults, so this configuration is optional.
type RepositoryConfig struct {
	// CacheSize is the LRU cache size in bytes for job metadata and energy footprints.
	// Default: 1MB (1024 * 1024 bytes)
	CacheSize int

	// MaxOpenConnections is the maximum number of open database connections.
	// Default: 4
	MaxOpenConnections int

	// MaxIdleConnections is the maximum number of idle database connections.
	// Default: 4
	MaxIdleConnections int

	// ConnectionMaxLifetime is the maximum amount of time a connection may be reused.
	// Default: 1 hour
	ConnectionMaxLifetime time.Duration

	// ConnectionMaxIdleTime is the maximum amount of time a connection may be idle.
	// Default: 1 hour
	ConnectionMaxIdleTime time.Duration

	// MinRunningJobDuration is the minimum duration in seconds for a job to be
	// considered in "running jobs" queries. This filters out very short jobs.
	// Default: 600 seconds (10 minutes)
	MinRunningJobDuration int
}

// DefaultConfig returns the default repository configuration.
// These values are optimized for typical deployments.
func DefaultConfig() *RepositoryConfig {
	return &RepositoryConfig{
		CacheSize:             1 * 1024 * 1024, // 1MB
		MaxOpenConnections:    4,
		MaxIdleConnections:    4,
		ConnectionMaxLifetime: time.Hour,
		ConnectionMaxIdleTime: time.Hour,
		MinRunningJobDuration: 600, // 10 minutes
	}
}

// repoConfig is the package-level configuration instance.
// It is initialized with defaults and can be overridden via SetConfig.
var repoConfig *RepositoryConfig = DefaultConfig()

// SetConfig sets the repository configuration.
// This must be called before any repository initialization (Connect, GetJobRepository, etc.).
// If not called, default values from DefaultConfig() are used.
func SetConfig(cfg *RepositoryConfig) {
	if cfg != nil {
		repoConfig = cfg
	}
}

// GetConfig returns the current repository configuration.
func GetConfig() *RepositoryConfig {
	return repoConfig
}


The user reverted the `DbCacheSizeMB`/`DbSoftHeapLimitMB` fields from config.go, the `setupSqlite` signature change, the `_cache_size` change, the `DbConfig` struct, the schema, and the README. It looks like the user reverted all my changes and also made their own modifications (e.g., `_cache_size` is now `1000000000`, `ConnectionMaxIdleTime` default changed to `time.Hour`, soft_heap_limit pragma removed entirely). Let me check main.go and the other files too:


     1→// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
     2→// All rights reserved. This file is part of cc-backend.
     3→// Use of this source code is governed by a MIT-style
     4→// license that can be found in the LICENSE file.
     5→
     6→package repository
     7→
     8→import (
     9→	"database/sql"
    10→	"fmt"
    11→	"net/url"
    12→	"sync"
    13→	"time"
    14→
    15→	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
    16→	"github.com/jmoiron/sqlx"
    17→	"github.com/mattn/go-sqlite3"
    18→	"github.com/qustavo/sqlhooks/v2"
    19→)
    20→
    21→var (
    22→	dbConnOnce     sync.Once
    23→	dbConnInstance *DBConnection
    24→)
    25→
    26→type DBConnection struct {
    27→	DB     *sqlx.DB
    28→	Driver string
    29→}
    30→
    31→type DatabaseOptions struct {
    32→	URL                   string
    33→	MaxOpenConnections    int
    34→	MaxIdleConnections    int
    35→	ConnectionMaxLifetime time.Duration
    36→	ConnectionMaxIdleTime time.Duration
    37→}
    38→
    39→func setupSqlite(db *sql.DB) error {
    40→	pragmas := []string{
    41→		"temp_store = memory",
    42→	}
    43→
    44→	for _, pragma := range pragmas {
    45→		_, err := db.Exec("PRAGMA " + pragma)
    46→		if err != nil {
    47→			return err
    48→		}
    49→	}
    50→
    51→	// Update query planner statistics so SQLite picks optimal indexes.
    52→	// Without this, SQLite guesses row distributions and often chooses wrong
    53→	// indexes for queries with IN clauses + ORDER BY, causing full table sorts
    54→	// in temp B-trees instead of using covering indexes.
    55→	if _, err := db.Exec("ANALYZE"); err != nil {
    56→		cclog.Warnf("Failed to run ANALYZE: %v", err)
    57→	}
    58→
    59→	return nil
    60→}
    61→
    62→func Connect(db string) {
    63→	var err error
    64→	var dbHandle *sqlx.DB
    65→
    66→	dbConnOnce.Do(func() {
    67→		opts := DatabaseOptions{
    68→			URL:                   db,
    69→			MaxOpenConnections:    repoConfig.MaxOpenConnections,
    70→			MaxIdleConnections:    repoConfig.MaxIdleConnections,
    71→			ConnectionMaxLifetime: repoConfig.ConnectionMaxLifetime,
    72→			ConnectionMaxIdleTime: repoConfig.ConnectionMaxIdleTime,
    73→		}
    74→
    75→		// TODO: Have separate DB handles for Writes and Reads
    76→		// Optimize SQLite connection: https://kerkour.com/sqlite-for-servers
    77→		connectionURLParams := make(url.Values)
    78→		connectionURLParams.Add("_txlock", "immediate")
    79→		connectionURLParams.Add("_journal_mode", "WAL")
    80→		connectionURLParams.Add("_busy_timeout", "5000")
    81→		connectionURLParams.Add("_synchronous", "NORMAL")
    82→		connectionURLParams.Add("_cache_size", "1000000000")
    83→		connectionURLParams.Add("_foreign_keys", "true")
    84→		opts.URL = fmt.Sprintf("file:%s?%s", opts.URL, connectionURLParams.Encode())
    85→
    86→		if cclog.Loglevel() == "debug" {
    87→			sql.Register("sqlite3WithHooks", sqlhooks.Wrap(&sqlite3.SQLiteDriver{}, &Hooks{}))
    88→			dbHandle, err = sqlx.Open("sqlite3WithHooks", opts.URL)
    89→		} else {
    90→			dbHandle, err = sqlx.Open("sqlite3", opts.URL)
    91→		}
    92→
    93→		if err != nil {
    94→			cclog.Abortf("DB Connection: Could not connect to SQLite database with sqlx.Open().\nError: %s\n", err.Error())
    95→		}
    96→
    97→		err = setupSqlite(dbHandle.DB)
    98→		if err != nil {
    99→			cclog.Abortf("Failed sqlite db setup.\nError: %s\n", err.Error())
   100→		}
   101→
   102→		dbHandle.SetMaxOpenConns(opts.MaxOpenConnections)
   103→		dbHandle.SetMaxIdleConns(opts.MaxIdleConnections)
   104→		dbHandle.SetConnMaxLifetime(opts.ConnectionMaxLifetime)
   105→		dbHandle.SetConnMaxIdleTime(opts.ConnectionMaxIdleTime)
   106→
   107→		dbConnInstance = &DBConnection{DB: dbHandle}
   108→		err = checkDBVersion(dbHandle.DB)
   109→		if err != nil {
   110→			cclog.Abortf("DB Connection: Failed DB version check.\nError: %s\n", err.Error())
   111→		}
   112→	})
   113→}
   114→
   115→func GetConnection() *DBConnection {
   116→	if dbConnInstance == nil {
   117→		cclog.Fatalf("Database connection not initialized!")
   118→	}
   119→
   120→	return dbConnInstance
   121→}
   122→
   123→// ResetConnection closes the current database connection and resets the connection state.
   124→// This function is intended for testing purposes only to allow test isolation.
   125→func ResetConnection() error {
   126→	if dbConnInstance != nil && dbConnInstance.DB != nil {
   127→		if err := dbConnInstance.DB.Close(); err != nil {
   128→			return fmt.Errorf("failed to close database connection: %w", err)
   129→		}
   130→	}
   131→
   132→	dbConnInstance = nil
   133→	dbConnOnce = sync.Once{}
   134→	jobRepoInstance = nil
   135→	jobRepoOnce = sync.Once{}
   136→	nodeRepoInstance = nil
   137→	nodeRepoOnce = sync.Once{}
   138→	userRepoInstance = nil
   139→	userRepoOnce = sync.Once{}
   140→	userCfgRepoInstance = nil
   141→	userCfgRepoOnce = sync.Once{}
   142→
   143→	return nil
   144→}
   145→

// Copyright (C) NHR@FAU, University Erlangen-Nuremberg.
// All rights reserved. This file is part of cc-backend.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.

package repository

import (
	"database/sql"
	"fmt"
	"net/url"
	"sync"
	"time"

	cclog "github.com/ClusterCockpit/cc-lib/v2/ccLogger"
	"github.com/jmoiron/sqlx"
	"github.com/mattn/go-sqlite3"
	"github.com/qustavo/sqlhooks/v2"
)

var (
	dbConnOnce     sync.Once
	dbConnInstance *DBConnection
)

type DBConnection struct {
	DB     *sqlx.DB
	Driver string
}

type DatabaseOptions struct {
	URL                   string
	MaxOpenConnections    int
	MaxIdleConnections    int
	ConnectionMaxLifetime time.Duration
	ConnectionMaxIdleTime time.Duration
}

func setupSqlite(db *sql.DB) error {
	pragmas := []string{
		"temp_store = memory",
	}

	for _, pragma := range pragmas {
		_, err := db.Exec("PRAGMA " + pragma)
		if err != nil {
			return err
		}
	}

	// Update query planner statistics so SQLite picks optimal indexes.
	// Without this, SQLite guesses row distributions and often chooses wrong
	// indexes for queries with IN clauses + ORDER BY, causing full table sorts
	// in temp B-trees instead of using covering indexes.
	if _, err := db.Exec("ANALYZE"); err != nil {
		cclog.Warnf("Failed to run ANALYZE: %v", err)
	}

	return nil
}

func Connect(db string) {
	var err error
	var dbHandle *sqlx.DB

	dbConnOnce.Do(func() {
		opts := DatabaseOptions{
			URL:                   db,
			MaxOpenConnections:    repoConfig.MaxOpenConnections,
			MaxIdleConnections:    repoConfig.MaxIdleConnections,
			ConnectionMaxLifetime: repoConfig.ConnectionMaxLifetime,
			ConnectionMaxIdleTime: repoConfig.ConnectionMaxIdleTime,
		}

		// TODO: Have separate DB handles for Writes and Reads
		// Optimize SQLite connection: https://kerkour.com/sqlite-for-servers
		connectionURLParams := make(url.Values)
		connectionURLParams.Add("_txlock", "immediate")
		connectionURLParams.Add("_journal_mode", "WAL")
		connectionURLParams.Add("_busy_timeout", "5000")
		connectionURLParams.Add("_synchronous", "NORMAL")
		connectionURLParams.Add("_cache_size", "1000000000")
		connectionURLParams.Add("_foreign_keys", "true")
		opts.URL = fmt.Sprintf("file:%s?%s", opts.URL, connectionURLParams.Encode())

		if cclog.Loglevel() == "debug" {
			sql.Register("sqlite3WithHooks", sqlhooks.Wrap(&sqlite3.SQLiteDriver{}, &Hooks{}))
			dbHandle, err = sqlx.Open("sqlite3WithHooks", opts.URL)
		} else {
			dbHandle, err = sqlx.Open("sqlite3", opts.URL)
		}

		if err != nil {
			cclog.Abortf("DB Connection: Could not connect to SQLite database with sqlx.Open().\nError: %s\n", err.Error())
		}

		err = setupSqlite(dbHandle.DB)
		if err != nil {
			cclog.Abortf("Failed sqlite db setup.\nError: %s\n", err.Error())
		}

		dbHandle.SetMaxOpenConns(opts.MaxOpenConnections)
		dbHandle.SetMaxIdleConns(opts.MaxIdleConnections)
		dbHandle.SetConnMaxLifetime(opts.ConnectionMaxLifetime)
		dbHandle.SetConnMaxIdleTime(opts.ConnectionMaxIdleTime)

		dbConnInstance = &DBConnection{DB: dbHandle}
		err = checkDBVersion(dbHandle.DB)
		if err != nil {
			cclog.Abortf("DB Connection: Failed DB version check.\nError: %s\n", err.Error())
		}
	})
}

func GetConnection() *DBConnection {
	if dbConnInstance == nil {
		cclog.Fatalf("Database connection not initialized!")
	}

	return dbConnInstance
}

// ResetConnection closes the current database connection and resets the connection state.
// This function is intended for testing purposes only to allow test isolation.
func ResetConnection() error {
	if dbConnInstance != nil && dbConnInstance.DB != nil {
		if err := dbConnInstance.DB.Close(); err != nil {
			return fmt.Errorf("failed to close database connection: %w", err)
		}
	}

	dbConnInstance = nil
	dbConnOnce = sync.Once{}
	jobRepoInstance = nil
	jobRepoOnce = sync.Once{}
	nodeRepoInstance = nil
	nodeRepoOnce = sync.Once{}
	userRepoInstance = nil
	userRepoOnce = sync.Once{}
	userCfgRepoInstance = nil
	userCfgRepoOnce = sync.Once{}

	return nil
}


   110→func initDatabase() error {
   111→	repository.Connect(config.Keys.DB)
   112→	return nil
   113→}
   114→
   115→func handleDatabaseCommands() error {
   116→	if flagMigrateDB {
   117→		err := repository.MigrateDB(config.Keys.DB)
   118→		if err != nil {
   119→			return fmt.Errorf("migrating database to version %d: %w", repository.Version, err)

func initDatabase() error {
	repository.Connect(config.Keys.DB)
	return nil
}

func handleDatabaseCommands() error {
	if flagMigrateDB {
		err := repository.MigrateDB(config.Keys.DB)
		if err != nil {
			return fmt.Errorf("migrating database to version %d: %w", repository.Version, err)


    78→	// Node state retention configuration
    79→	NodeStateRetention *NodeStateRetention `json:"nodestate-retention"`
    80→}
    81→
    82→type NodeStateRetention struct {
    83→	Policy             string `json:"policy"`      // "delete" or "move"
    84→	Age                int    `json:"age"`         // hours, default 24
    85→	TargetKind         string `json:"target-kind"` // "file" or "s3"
    86→	TargetPath         string `json:"target-path"`
    87→	TargetEndpoint     string `json:"target-endpoint"`
    88→	TargetBucket       string `json:"target-bucket"`
    89→	TargetAccessKey    string `json:"target-access-key"`
    90→	TargetSecretKey    string `json:"target-secret-key"`
    91→	TargetRegion       string `json:"target-region"`
    92→	TargetUsePathStyle bool   `json:"target-use-path-style"`

	// Node state retention configuration
	NodeStateRetention *NodeStateRetention `json:"nodestate-retention"`
}

type NodeStateRetention struct {
	Policy             string `json:"policy"`      // "delete" or "move"
	Age                int    `json:"age"`         // hours, default 24
	TargetKind         string `json:"target-kind"` // "file" or "s3"
	TargetPath         string `json:"target-path"`
	TargetEndpoint     string `json:"target-endpoint"`
	TargetBucket       string `json:"target-bucket"`
	TargetAccessKey    string `json:"target-access-key"`
	TargetSecretKey    string `json:"target-secret-key"`
	TargetRegion       string `json:"target-region"`
	TargetUsePathStyle bool   `json:"target-use-path-style"`


   175→          "description": "Maximum parquet file size in MB (default: 128).",
   176→          "type": "integer"
   177→        }
   178→      },
   179→      "required": ["policy"]
   180→    }
   181→  }
   182→}`
   183→

          "description": "Maximum parquet file size in MB (default: 128).",
          "type": "integer"
        }
      },
      "required": ["policy"]
    }
  }
}`


All my previous changes were reverted by the user. I'll re-add the README section with the improved clarification about per-connection caches.


Edit

## Database Configuration

cc-backend uses SQLite as its database. For large installations, SQLite memory
usage can be tuned via the optional `db-config` section in config.json under
`main`:

```json
{
  "main": {
    "db": "./var/job.db",
    "db-config": {
      "cache-size-mb": 2048,
      "soft-heap-limit-mb": 16384,
      "max-open-connections": 4,
      "max-idle-connections": 4,
      "max-idle-time-minutes": 10
    }
  }
}
```

All fields are optional. If `db-config` is omitted entirely, built-in defaults
are used.

### Options

| Option | Default | Description |
|--------|---------|-------------|
| `cache-size-mb` | 2048 | SQLite page cache size per connection in MB. Maps to `PRAGMA cache_size`. Total cache memory is up to `cache-size-mb × max-open-connections`. |
| `soft-heap-limit-mb` | 16384 | Process-wide SQLite soft heap limit in MB. SQLite will try to release cache pages to stay under this limit. Queries won't fail if exceeded, but cache eviction becomes more aggressive. |
| `max-open-connections` | 4 | Maximum number of open database connections. |
| `max-idle-connections` | 4 | Maximum number of idle database connections kept in the pool. |
| `max-idle-time-minutes` | 10 | Maximum time in minutes a connection can sit idle before being closed. |

### Sizing Guidelines

SQLite's `cache_size` is a **per-connection** setting — each connection
maintains its own independent page cache. With multiple connections, the total
memory available for caching is the sum across all connections.

In practice, different connections tend to cache **different pages** (e.g., one
handles a job listing query while another runs a statistics aggregation), so
their caches naturally spread across the database. The formula
`DB_size / max-open-connections` gives enough per-connection cache that the
combined caches can cover the entire database.

However, this is a best-case estimate. Connections running similar queries will
cache the same pages redundantly. In the worst case (all connections caching
identical pages), only `cache-size-mb` worth of unique data is cached rather
than `cache-size-mb × max-open-connections`. For workloads with diverse
concurrent queries, cache overlap is typically low.

**Rules of thumb:**

- **cache-size-mb**: Set to `DB_size_in_MB / max-open-connections` to allow the
  entire database to be cached in memory. For example, an 80GB database with 8
  connections needs at least 10240 MB (10GB) per connection. If your workload
  has many similar concurrent queries, consider setting it higher to account for
  cache overlap between connections.
- **soft-heap-limit-mb**: Should be >= `cache-size-mb × max-open-connections` to
  avoid cache thrashing. This is the total SQLite memory budget for the process.
- On small installations the defaults work well. On servers with large databases
  (tens of GB) and plenty of RAM, increasing these values significantly improves
  query performance by reducing disk I/O.

### Example: Large Server (512GB RAM, 80GB database)

```json
{
  "main": {
    "db-config": {
      "cache-size-mb": 16384,
      "soft-heap-limit-mb": 131072,
      "max-open-connections": 8,
      "max-idle-time-minutes": 30
    }
  }
}
```

This allows the entire 80GB database to be cached (8 × 16GB = 128GB page cache)
with a 128GB soft heap limit, using about 25% of available RAM.

The effective configuration is logged at startup for verification.

## Project file structure


The file /Users/jan/prg/CC/cc-backend/README.md has been updated successfully.

## Database Configuration

cc-backend uses SQLite as its database. For large installations, SQLite memory
usage can be tuned via the optional `db-config` section in config.json under
`main`:

```json
{
  "main": {
    "db": "./var/job.db",
    "db-config": {
      "cache-size-mb": 2048,
      "soft-heap-limit-mb": 16384,
      "max-open-connections": 4,
      "max-idle-connections": 4,
      "max-idle-time-minutes": 10
    }
  }
}
```

All fields are optional. If `db-config` is omitted entirely, built-in defaults
are used.

### Options

| Option | Default | Description |
|--------|---------|-------------|
| `cache-size-mb` | 2048 | SQLite page cache size per connection in MB. Maps to `PRAGMA cache_size`. Total cache memory is up to `cache-size-mb × max-open-connections`. |
| `soft-heap-limit-mb` | 16384 | Process-wide SQLite soft heap limit in MB. SQLite will try to release cache pages to stay under this limit. Queries won't fail if exceeded, but cache eviction becomes more aggressive. |
| `max-open-connections` | 4 | Maximum number of open database connections. |
| `max-idle-connections` | 4 | Maximum number of idle database connections kept in the pool. |
| `max-idle-time-minutes` | 10 | Maximum time in minutes a connection can sit idle before being closed. |

### Sizing Guidelines

SQLite's `cache_size` is a **per-connection** setting — each connection
maintains its own independent page cache. With multiple connections, the total
memory available for caching is the sum across all connections.

In practice, different connections tend to cache **different pages** (e.g., one
handles a job listing query while another runs a statistics aggregation), so
their caches naturally spread across the database. The formula
`DB_size / max-open-connections` gives enough per-connection cache that the
combined caches can cover the entire database.

However, this is a best-case estimate. Connections running similar queries will
cache the same pages redundantly. In the worst case (all connections caching
identical pages), only `cache-size-mb` worth of unique data is cached rather
than `cache-size-mb × max-open-connections`. For workloads with diverse
concurrent queries, cache overlap is typically low.

**Rules of thumb:**

- **cache-size-mb**: Set to `DB_size_in_MB / max-open-connections` to allow the
  entire database to be cached in memory. For example, an 80GB database with 8
  connections needs at least 10240 MB (10GB) per connection. If your workload
  has many similar concurrent queries, consider setting it higher to account for
  cache overlap between connections.
- **soft-heap-limit-mb**: Should be >= `cache-size-mb × max-open-connections` to
  avoid cache thrashing. This is the total SQLite memory budget for the process.
- On small installations the defaults work well. On servers with large databases
  (tens of GB) and plenty of RAM, increasing these values significantly improves
  query performance by reducing disk I/O.

### Example: Large Server (512GB RAM, 80GB database)

```json
{
  "main": {
    "db-config": {
      "cache-size-mb": 16384,
      "soft-heap-limit-mb": 131072,
      "max-open-connections": 8,
      "max-idle-time-minutes": 30
    }
  }
}
```

This allows the entire 80GB database to be cached (8 × 16GB = 128GB page cache)
with a 128GB soft heap limit, using about 25% of available RAM.

The effective configuration is logged at startup for verification.

## Project file structure

# NOTE

While we do our best to keep the master branch in a usable state, there is no guarantee the master branch works.
Please do not use it for production!

Please have a look at the [Release
Notes](https://github.com/ClusterCockpit/cc-backend/blob/master/ReleaseNotes.md)
for breaking changes!

# ClusterCockpit REST and GraphQL API backend

[![Build](https://github.com/ClusterCockpit/cc-backend/actions/workflows/test.yml/badge.svg)](https://github.com/ClusterCockpit/cc-backend/actions/workflows/test.yml)

This is a Golang backend implementation for a REST and GraphQL API according to
the [ClusterCockpit
specifications](https://github.com/ClusterCockpit/cc-specifications). It also
includes a web interface for ClusterCockpit. This implementation replaces the
previous PHP Symfony based ClusterCockpit web interface. The reasons for
switching from PHP Symfony to a Golang based solution are explained
[here](https://github.com/ClusterCockpit/ClusterCockpit/wiki/Why-we-switched-from-PHP-Symfony-to-a-Golang-based-solution).

## Overview

This is a Golang web backend for the ClusterCockpit job-specific performance
monitoring framework. It provides a REST API and an optional NATS-based messaging
API for integrating ClusterCockpit with an HPC cluster batch system and external
analysis scripts. Data exchange between the web front-end and the back-end is
based on a GraphQL API. The web frontend is also served by the backend using
[Svelte](https://svelte.dev/) components. Layout and styling are based on
[Bootstrap 5](https://getbootstrap.com/) using
[Bootstrap Icons](https://icons.getbootstrap.com/).

The backend uses [SQLite 3](https://sqlite.org/) as the relational SQL database.
While there are metric data backends for the InfluxDB and Prometheus time series
databases, the only tested and supported setup is to use cc-metric-store as the
metric data backend. Documentation on how to integrate ClusterCockpit with other
time series databases will be added in the future.

For real-time integration with HPC systems, the backend can subscribe to
[NATS](https://nats.io/) subjects to receive job start/stop events and node
state updates, providing an alternative to REST API polling.

Completed batch jobs are stored in a file-based job archive according to
[this specification](https://github.com/ClusterCockpit/cc-specifications/tree/master/job-archive).
The backend supports authentication via local accounts, an external LDAP
directory, and JWT tokens. Authorization for APIs is implemented with
[JWT](https://jwt.io/) tokens created with public/private key encryption.

You find a detailed documentation on the [ClusterCockpit
Webpage](https://clustercockpit.org).

## Build requirements

ClusterCockpit requires a current version of the golang toolchain and node.js.
You can check `go.mod` to see what is the current minimal golang version needed.
Homebrew and Archlinux usually have current golang versions. For other Linux
distros this often means that you have to install the golang compiler yourself.
Fortunately, this is easy with golang. Since much of the functionality is based
on the Go standard library, it is crucial for security and performance to use a
current version of golang. In addition, an old golang toolchain may limit the supported
versions of third-party packages.

## How to try ClusterCockpit with a demo setup

We provide a shell script that downloads demo data and automatically starts the
cc-backend. You will need `wget`, `go`, `node`, `npm` in your path to
start the demo. The demo downloads 32MB of data (223MB on disk).

```sh
git clone https://github.com/ClusterCockpit/cc-backend.git
cd ./cc-backend
./startDemo.sh
```

You can also try the demo using the latest release binary.
Create a folder and put the release binary `cc-backend` into this folder.
Execute the following steps:

```shell
./cc-backend -init
vim config.json (Add a second cluster entry and name the clusters alex and fritz)
wget https://hpc-mover.rrze.uni-erlangen.de/HPC-Data/0x7b58aefb/eig7ahyo6fo2bais0ephuf2aitohv1ai/job-archive-demo.tar
tar xf job-archive-demo.tar
./cc-backend -init-db -add-user demo:admin:demo -loglevel info
./cc-backend -server -dev -loglevel info
```

You can access the web interface at [http://localhost:8080](http://localhost:8080).
Credentials for login are `demo:demo`.
Please note that some views do not work without a metric backend (e.g., the
Analysis, Systems and Status views).

## How to build and run

There is a Makefile to automate the build of cc-backend. The Makefile supports
the following targets:

- `make`: Initialize `var` directory and build svelte frontend and backend
  binary. Note that there is no proper prerequisite handling. Any change of
  frontend source files will result in a complete rebuild.
- `make clean`: Clean go build cache and remove binary.
- `make test`: Run the tests that are also run in the GitHub workflow setup.

A common workflow for setting up cc-backend from scratch is:

```sh
git clone https://github.com/ClusterCockpit/cc-backend.git

# Build binary
cd ./cc-backend/
make

# EDIT THE .env FILE BEFORE YOU DEPLOY (Change the secrets)!
# If authentication is disabled, it can be empty.
cp configs/env-template.txt  .env
vim .env

cp configs/config.json .
vim config.json

#Optional: Link an existing job archive:
ln -s <your-existing-job-archive> ./var/job-archive

# This will first initialize the job.db database by traversing all
# `meta.json` files in the job-archive and add a new user.
./cc-backend -init-db -add-user <your-username>:admin:<your-password>

# Start a HTTP server (HTTPS can be enabled in the configuration, the default port is 8080).
# The --dev flag enables GraphQL Playground (http://localhost:8080/playground) and Swagger UI (http://localhost:8080/swagger).
./cc-backend -server  -dev

# Show other options:
./cc-backend -help
```

## Project file structure

- [`.github/`](https://github.com/ClusterCockpit/cc-backend/tree/master/.github)
  GitHub Actions workflows and dependabot configuration for CI/CD.
- [`api/`](https://github.com/ClusterCockpit/cc-backend/tree/master/api)
  contains the API schema files for the REST and GraphQL APIs. The REST API is
  documented in the OpenAPI 3.0 format in
  [./api/swagger.yaml](./api/swagger.yaml). The GraphQL schema is in
  [./api/schema.graphqls](./api/schema.graphqls).
- [`cmd/cc-backend`](https://github.com/ClusterCockpit/cc-backend/tree/master/cmd/cc-backend)
  contains the main application entry point and CLI implementation.
- [`configs/`](https://github.com/ClusterCockpit/cc-backend/tree/master/configs)
  contains documentation about configuration and command line options and required
  environment variables. Sample configuration files are provided.
- [`init/`](https://github.com/ClusterCockpit/cc-backend/tree/master/init)
  contains an example of setting up systemd for production use.
- [`internal/`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal)
  contains library source code that is not intended for use by others.
  - [`api`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/api)
    REST API handlers and NATS integration
  - [`archiver`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/archiver)
    Job archiving functionality
  - [`auth`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/auth)
    Authentication (local, LDAP, OIDC) and JWT token handling
  - [`config`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/config)
    Configuration management and validation
  - [`graph`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/graph)
    GraphQL schema and resolvers
  - [`importer`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/importer)
    Job data import and database initialization
  - [`metricdispatch`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricdispatch)
    Dispatches metric data loading to appropriate backends
  - [`repository`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/repository)
    Database repository layer for jobs and metadata
  - [`routerConfig`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/routerConfig)
    HTTP router configuration and middleware
  - [`tagger`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/tagger)
    Job classification and application detection
  - [`taskmanager`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/taskmanager)
    Background task management and scheduled jobs
  - [`metricstoreclient`](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/metricstoreclient)
    Client for cc-metric-store queries
- [`pkg/`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg)
  contains Go packages that can be used by other projects.
  - [`archive`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/archive)
    Job archive backend implementations (filesystem, S3, SQLite)
  - [`metricstore`](https://github.com/ClusterCockpit/cc-backend/tree/master/pkg/metricstore)
    In-memory metric data store with checkpointing and metric loading
- [`tools/`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools)
  Additional command line helper tools.
  - [`archive-manager`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-manager)
    Commands for getting infos about an existing job archive, importing jobs
    between archive backends, and converting archives between JSON and Parquet formats.
  - [`archive-migration`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/archive-migration)
    Tool for migrating job archives between formats.
  - [`convert-pem-pubkey`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/convert-pem-pubkey)
    Tool to convert external pubkey for use in `cc-backend`.
  - [`gen-keypair`](https://github.com/ClusterCockpit/cc-backend/tree/master/tools/gen-keypair)
    contains a small application to generate a compatible JWT keypair. You find
    documentation on how to use it
    [here](https://github.com/ClusterCockpit/cc-backend/blob/master/docs/JWT-Handling.md).
- [`web/`](https://github.com/ClusterCockpit/cc-backend/tree/master/web)
  Server-side templates and frontend-related files:
  - [`frontend`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/frontend)
    Svelte components and static assets for the frontend UI
  - [`templates`](https://github.com/ClusterCockpit/cc-backend/tree/master/web/templates)
    Server-side Go templates, including monitoring views
- [`gqlgen.yml`](https://github.com/ClusterCockpit/cc-backend/blob/master/gqlgen.yml)
  Configures the behaviour and generation of
  [gqlgen](https://github.com/99designs/gqlgen).
- [`startDemo.sh`](https://github.com/ClusterCockpit/cc-backend/blob/master/startDemo.sh)
  is a shell script that sets up demo data, and builds and starts `cc-backend`.

+cc-backend uses SQLite as its database. For large installations, SQLite memory

+usage can be tuned via the optional `db-config` section in config.json under

+All fields are optional. If `db-config` is omitted entirely, built-in defaults

+| `cache-size-mb` | 2048 | SQLite page cache size per connection in MB. Maps to `PRAGMA cache_size`. Total cache memory is up to `cache-size-mb × max-open-connections`. |

+| `soft-heap-limit-mb` | 16384 | Process-wide SQLite soft heap limit in MB. SQLite will try to release cache pages to stay under this limit. Queries won't fail if exceeded, but cache eviction becomes more aggressive. |

+| `max-open-connections` | 4 | Maximum number of open database connections. |

+| `max-idle-connections` | 4 | Maximum number of idle database connections kept in the pool. |

+| `max-idle-time-minutes` | 10 | Maximum time in minutes a connection can sit idle before being closed. |

+SQLite's `cache_size` is a **per-connection** setting — each connection

+maintains its own independent page cache. With multiple connections, the total

+memory available for caching is the sum across all connections.

+In practice, different connections tend to cache **different pages** (e.g., one

+handles a job listing query while another runs a statistics aggregation), so

+their caches naturally spread across the database. The formula

+`DB_size / max-open-connections` gives enough per-connection cache that the

+combined caches can cover the entire database.

+However, this is a best-case estimate. Connections running similar queries will

+cache the same pages redundantly. In the worst case (all connections caching

+identical pages), only `cache-size-mb` worth of unique data is cached rather

+than `cache-size-mb × max-open-connections`. For workloads with diverse

+concurrent queries, cache overlap is typically low.

+- **cache-size-mb**: Set to `DB_size_in_MB / max-open-connections` to allow the

+  entire database to be cached in memory. For example, an 80GB database with 8

+  connections needs at least 10240 MB (10GB) per connection. If your workload

+  has many similar concurrent queries, consider setting it higher to account for

+- **soft-heap-limit-mb**: Should be >= `cache-size-mb × max-open-connections` to

+  avoid cache thrashing. This is the total SQLite memory budget for the process.

+- On small installations the defaults work well. On servers with large databases

+  (tens of GB) and plenty of RAM, increasing these values significantly improves

+  query performance by reducing disk I/O.

+### Example: Large Server (512GB RAM, 80GB database)

+This allows the entire 80GB database to be cached (8 × 16GB = 128GB page cache)

+with a 128GB soft heap limit, using about 25% of available RAM.

+The effective configuration is logged at startup for verification.


Added the database configuration section with a detailed clarification in the "Sizing Guidelines" subsection explaining that `cache_size` is per-connection, how caches naturally spread across different pages with diverse queries, and that the `DB_size / max-open-connections` formula is a best-case estimate — with a note to set it higher if concurrent queries overlap significantly.


Yes please add a clarification to the README


<local-command-caveat>Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.</local-command-caveat>


<command-name>/exit</command-name>
            <command-message>exit</command-message>
            <command-args></command-args>


<local-command-stdout>Catch you later!</local-command-stdout>