Commit Graph

15 Commits

Author SHA1 Message Date
8234ad3126 fix: Fix metricstore memory explosion from broken emergency free and batch aborts
- Fix MemoryUsageTracker: remove premature bufferPool.Clear() that prevented
  mem.Alloc from decreasing, replace broken ForceFree loop (100 iterations
  with no GC) with progressive time-based Free at 75%/50%/25% retention,
  add bufferPool.Clear()+GC between steps so memory stats update correctly
- Enable debug.FreeOSMemory() after emergency freeing to return memory to OS
- Add adaptive ticker: 30s checks when memory >80% of cap, normal otherwise
- Reduce default memory check interval from 1h to 5min
- Don't abort entire NATS batch on single write error (out-of-order timestamp),
  log warning and continue processing remaining lines
- Prune empty levels from tree after free() to reduce overhead
- Include buffer struct overhead in sizeInBytes() for more accurate reporting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 7ce28627fc1d
2026-03-13 07:57:35 +01:00
26982088c3 Consolidate code for external and internal ccms buildQueries function
Entire-Checkpoint: fc3be444ef4c
2026-03-04 16:43:05 +01:00
Aditya Ujeniya
a243e17499 Update to shutdown worker for WAL checkpointing mode 2026-03-02 15:27:06 +01:00
1ec41d8389 Review and improve buffer pool implmentation. Add unit tests. 2026-02-28 19:34:33 +01:00
Aditya Ujeniya
07b989cb81 Add new bufferPool implementation 2026-02-27 14:44:32 +01:00
a418abc7d5 Run go fix 2026-02-27 14:40:26 +01:00
ca0f9a42c7 Introduce metric store binary checkpoints with write ahead log 2026-02-26 10:08:40 +01:00
Aditya Ujeniya
7101d2bb3b Handle the metric/host not found case differently 2026-01-28 17:47:38 +01:00
0d857b49a2 Disable explicit GC calls 2026-01-28 11:21:27 +01:00
eb5aa9ad02 Disable explicit GC calls 2026-01-28 11:21:02 +01:00
9d15a87c88 Take into account the real allocated heap memory in MemoryUsageTracker 2026-01-27 18:23:09 +01:00
bbde91a1f9 Move wg increment inside goroutines. Make GC calls less aggressive 2026-01-27 17:25:29 +01:00
752e19c276 Pull out metric List build from metricstore Init 2026-01-27 17:06:52 +01:00
c782043c64 Upgrade cclib and remove usage of obsolete util.Float 2026-01-26 08:38:53 +01:00
f41301036b Move metricstore from internal to pkg 2026-01-23 07:49:47 +01:00