ClusterCockpit/cc-backend
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-backend synced 2026-03-15 04:17:30 +01:00
Code Issues Packages Releases Activity

Fixed merge errors

Browse Source 
Entire-Checkpoint: ddd4fa4a7bbb
This commit is contained in:
Jan Eitzinger
2026-03-11 05:09:38 +01:00
parent 1cf99206a9
commit f8831e7040
6 changed files with 69 additions and 291 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
.entire/metadata/42401d2e-7d1c-4c0e-abe6-356cb2d48747/context.md
View File

@@ -1,114 +1,18 @@
# Session Context
Session ID: 42401d2e-7d1c-4c0e-abe6-356cb2d48747
Commit Message: Implement the following plan:
# Fix Missing `rows.Close()` Memory Leaks
Commit Message: Check if the fixes are correctly merged in nodes.go
## Prompts
### Prompt 1
Implement the following plan:
# Fix Missing `rows.Close()` Memory Leaks in SQLite3 Queries
## Context
Production memory leaks traced to queries that do full table scans (e.g., job state list sorted by `start_time` on all jobs). The root cause is `sql.Rows` objects not being closed after query execution. In Go's `database/sql`, every `rows` returned by `.Query()` holds a database connection and associated buffers until `rows.Close()` is called. Without `defer rows.Close()`, these leak on every code path (both success and error returns).
## Findings
**22 total `.Query()` calls** across the repository layer. **15 have `defer rows.Close()`**. **7 do not** (listed below). Additionally, 1 `Queryx` call in `tags.go` is also missing close.
In `node.go`, `QueryNodes` and `QueryNodesWithMeta` have partial `rows.Close()` only in error paths but **not on the success path** and not via `defer`.
`CountStates` and `CountStatesTimed` in `node.go` also lack `defer rows.Close()` (same partial pattern as above for CountStates, none at all for CountStatesTimed).
## Changes Required
### 1. `internal/repository/stats.go` — 6 functions missing `defer rows.Close()`
Add `defer rows.Close()` immediately after the `if err != nil` check for each:
| Line | Function |
|------|----------|
| 233 | `JobsStatsGrouped` |
| 438 | `JobCountGrouped` |
| 494 | `AddJobCountGrouped` |
| 553 | `AddJobCount` |
| 753 | `jobsStatisticsHistogram` |
| 821 | `jobsDurationStatisticsHistogram` |
| 946 | `jobsMetricStatisticsHistogram` |
Pattern — after each `Query()` error check, add:
```go
rows, err := query.RunWith(r.DB).Query()
if err != nil {
...
return nil, err
}
defer rows.Close() // <-- ADD THIS
```
### 2. `internal/repository/tags.go` — 2 leaks in `CountTags()`
**Line 282**: `xrows` from `r.DB.Queryx(...)` — add `defer xrows.Close()` after error check.
**Line 333**: `rows` from `q.RunWith(r.stmtCache).Query()` — add `defer rows.Close()` after error check.
### 3. `internal/repository/tags.go` — 3 leaks in `GetTags`, `GetTagsDirect`, `getArchiveTags`
**Line 508** (`GetTags`): add `defer rows.Close()` after error check.
**Line 541** (`GetTagsDirect`): add `defer rows.Close()` after error check.
**Line 579** (`getArchiveTags`): add `defer rows.Close()` after error check.
### 4. `internal/repository/node.go` — 4 functions missing `defer rows.Close()`
**Line 363** (`QueryNodes`): Replace the manual `rows.Close()` in the error path with `defer rows.Close()` immediately after the error check. Remove the explicit `rows.Close()` call on line 375.
**Line 412** (`QueryNodesWithMeta`): Same pattern — add `defer rows.Close()` after error check, remove explicit `rows.Close()` on line 427.
**Line 558** (`CountStates`): Add `defer rows.Close()` after error check. Remove explicit `rows.Close()` on line 569.
**Line 620** (`CountStatesTimed`): Add `defer rows.Close()` after error check. Remove explicit `rows.Close()` on line 633.
## Summary of All Edits
| File | Function | Action |
|------|----------|--------|
| `stats.go:237` | `JobsStatsGrouped` | Add `defer rows.Close()` |
| `stats.go:442` | `JobCountGrouped` | Add `defer rows.Close()` |
| `stats.go:498` | `AddJobCountGrouped` | Add `defer rows.Close()` |
| `stats.go:557` | `AddJobCount` | Add `defer rows.Close()` |
| `stats.go:757` | `jobsStatisticsHistogram` | Add `defer rows.Close()` |
| `stats.go:825` | `jobsDurationStatisticsHistogram` | Add `defer rows.Close()` |
| `stats.go:950` | `jobsMetricStatisticsHistogram` | Add `defer rows.Close()` |
| `tags.go:284` | `CountTags` (xrows) | Add `defer xrows.Close()` |
| `tags.go:336` | `CountTags` (rows) | Add `defer rows.Close()` |
| `tags.go:513` | `GetTags` | Add `defer rows.Close()` |
| `tags.go:546` | `GetTagsDirect` | Add `defer rows.Close()` |
| `tags.go:584` | `getArchiveTags` | Add `defer rows.Close()` |
| `node.go:368` | `QueryNodes` | Add `defer rows.Close()`, remove manual close on L375 |
| `node.go:417` | `QueryNodesWithMeta` | Add `defer rows.Close()`, remove manual close on L427 |
| `node.go:563` | `CountStates` | Add `defer rows.Close()`, remove manual close on L569 |
| `node.go:625` | `CountStatesTimed` | Add `defer rows.Close()`, remove manual close on L633 |
## Verification
1. `go build ./...` — ensure it compiles
2. `go test ./internal/repository/...` — run repository tests
3. `go vet ./internal/repository/...` — static analysis
If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /Users/jan/.claude/projects/-Users-jan-prg-CC-cc-backend/28147033-ddc8-4056-b064-e0558fbc614e.jsonl
Check if the fixes are correctly merged in nodes.go
## Summary
All changes are already in place and verified:
All clean. Here's a summary of what was wrong and fixed in `node.go`:
- **Build**: passes
- **go vet**: passes
- **Tests**: pass
The changes are currently unstaged. All 16 locations from the plan have been fixed — `defer rows.Close()` added after each `Query()` error check, manual `rows.Close()` calls in error paths removed, and `rows.Err()` checks added where missing. The `AddJobCount` function was also simplified from `Query` + loop to `QueryRow().Scan()` since it returns a single value.
1. **Duplicate `QueryNodesWithMeta`** (lines 468-541): The old unfixed version was left in alongside the fixed version at line 391. Removed the duplicate.
2. **`CountStates`**: Had both old code (manual `stateMap` counting without SQL `COUNT(*)`) and new code (SQL `COUNT(*) ... GROUP BY`) concatenated together. The second `rows.Next()` loop was dead code since the first loop already consumed all rows. Replaced with the correct version using `SELECT column, COUNT(*) ... GROUP BY column`.
3. **`QueryNodes` line 376**: Leftover `rows.Close()` inside the scan error path (redundant with `defer`). Removed.

48
.entire/metadata/42401d2e-7d1c-4c0e-abe6-356cb2d48747/full.jsonl
View File

File diff suppressed because one or more lines are too long

94
.entire/metadata/42401d2e-7d1c-4c0e-abe6-356cb2d48747/prompt.txt
View File

@@ -1,93 +1 @@
Implement the following plan:
# Fix Missing `rows.Close()` Memory Leaks in SQLite3 Queries
## Context
Production memory leaks traced to queries that do full table scans (e.g., job state list sorted by `start_time` on all jobs). The root cause is `sql.Rows` objects not being closed after query execution. In Go's `database/sql`, every `rows` returned by `.Query()` holds a database connection and associated buffers until `rows.Close()` is called. Without `defer rows.Close()`, these leak on every code path (both success and error returns).
## Findings
**22 total `.Query()` calls** across the repository layer. **15 have `defer rows.Close()`**. **7 do not** (listed below). Additionally, 1 `Queryx` call in `tags.go` is also missing close.
In `node.go`, `QueryNodes` and `QueryNodesWithMeta` have partial `rows.Close()` only in error paths but **not on the success path** and not via `defer`.
`CountStates` and `CountStatesTimed` in `node.go` also lack `defer rows.Close()` (same partial pattern as above for CountStates, none at all for CountStatesTimed).
## Changes Required
### 1. `internal/repository/stats.go` — 6 functions missing `defer rows.Close()`
Add `defer rows.Close()` immediately after the `if err != nil` check for each:
| Line | Function |
|------|----------|
| 233 | `JobsStatsGrouped` |
| 438 | `JobCountGrouped` |
| 494 | `AddJobCountGrouped` |
| 553 | `AddJobCount` |
| 753 | `jobsStatisticsHistogram` |
| 821 | `jobsDurationStatisticsHistogram` |
| 946 | `jobsMetricStatisticsHistogram` |
Pattern — after each `Query()` error check, add:
```go
rows, err := query.RunWith(r.DB).Query()
if err != nil {
...
return nil, err
}
defer rows.Close() // <-- ADD THIS
```
### 2. `internal/repository/tags.go` — 2 leaks in `CountTags()`
**Line 282**: `xrows` from `r.DB.Queryx(...)` — add `defer xrows.Close()` after error check.
**Line 333**: `rows` from `q.RunWith(r.stmtCache).Query()` — add `defer rows.Close()` after error check.
### 3. `internal/repository/tags.go` — 3 leaks in `GetTags`, `GetTagsDirect`, `getArchiveTags`
**Line 508** (`GetTags`): add `defer rows.Close()` after error check.
**Line 541** (`GetTagsDirect`): add `defer rows.Close()` after error check.
**Line 579** (`getArchiveTags`): add `defer rows.Close()` after error check.
### 4. `internal/repository/node.go` — 4 functions missing `defer rows.Close()`
**Line 363** (`QueryNodes`): Replace the manual `rows.Close()` in the error path with `defer rows.Close()` immediately after the error check. Remove the explicit `rows.Close()` call on line 375.
**Line 412** (`QueryNodesWithMeta`): Same pattern — add `defer rows.Close()` after error check, remove explicit `rows.Close()` on line 427.
**Line 558** (`CountStates`): Add `defer rows.Close()` after error check. Remove explicit `rows.Close()` on line 569.
**Line 620** (`CountStatesTimed`): Add `defer rows.Close()` after error check. Remove explicit `rows.Close()` on line 633.
## Summary of All Edits
| File | Function | Action |
|------|----------|--------|
| `stats.go:237` | `JobsStatsGrouped` | Add `defer rows.Close()` |
| `stats.go:442` | `JobCountGrouped` | Add `defer rows.Close()` |
| `stats.go:498` | `AddJobCountGrouped` | Add `defer rows.Close()` |
| `stats.go:557` | `AddJobCount` | Add `defer rows.Close()` |
| `stats.go:757` | `jobsStatisticsHistogram` | Add `defer rows.Close()` |
| `stats.go:825` | `jobsDurationStatisticsHistogram` | Add `defer rows.Close()` |
| `stats.go:950` | `jobsMetricStatisticsHistogram` | Add `defer rows.Close()` |
| `tags.go:284` | `CountTags` (xrows) | Add `defer xrows.Close()` |
| `tags.go:336` | `CountTags` (rows) | Add `defer rows.Close()` |
| `tags.go:513` | `GetTags` | Add `defer rows.Close()` |
| `tags.go:546` | `GetTagsDirect` | Add `defer rows.Close()` |
| `tags.go:584` | `getArchiveTags` | Add `defer rows.Close()` |
| `node.go:368` | `QueryNodes` | Add `defer rows.Close()`, remove manual close on L375 |
| `node.go:417` | `QueryNodesWithMeta` | Add `defer rows.Close()`, remove manual close on L427 |
| `node.go:563` | `CountStates` | Add `defer rows.Close()`, remove manual close on L569 |
| `node.go:625` | `CountStatesTimed` | Add `defer rows.Close()`, remove manual close on L633 |
## Verification
1. `go build ./...` — ensure it compiles
2. `go test ./internal/repository/...` — run repository tests
3. `go vet ./internal/repository/...` — static analysis
If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /Users/jan/.claude/projects/-Users-jan-prg-CC-cc-backend/28147033-ddc8-4056-b064-e0558fbc614e.jsonl
Check if the fixes are correctly merged in nodes.go

10
.entire/metadata/42401d2e-7d1c-4c0e-abe6-356cb2d48747/summary.txt
View File

@@ -1,7 +1,5 @@
All changes are already in place and verified:
All clean. Here's a summary of what was wrong and fixed in `node.go`:
- **Build**: passes
- **go vet**: passes
- **Tests**: pass
The changes are currently unstaged. All 16 locations from the plan have been fixed — `defer rows.Close()` added after each `Query()` error check, manual `rows.Close()` calls in error paths removed, and `rows.Err()` checks added where missing. The `AddJobCount` function was also simplified from `Query` + loop to `QueryRow().Scan()` since it returns a single value.
1. **Duplicate `QueryNodesWithMeta`** (lines 468-541): The old unfixed version was left in alongside the fixed version at line 391. Removed the duplicate.
2. **`CountStates`**: Had both old code (manual `stateMap` counting without SQL `COUNT(*)`) and new code (SQL `COUNT(*) ... GROUP BY`) concatenated together. The second `rows.Next()` loop was dead code since the first loop already consumed all rows. Replaced with the correct version using `SELECT column, COUNT(*) ... GROUP BY column`.
3. **`QueryNodes` line 376**: Leftover `rows.Close()` inside the scan error path (redundant with `defer`). Removed.