ClusterCockpit/cc-backend
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-backend synced 2026-04-01 21:47:31 +02:00
Code Issues Packages Releases Activity
Files
entire/checkpoints/v1
cc-backend/34/d698f40bac/0/prompt.txt
Jan Eitzinger c827b09238 Checkpoint: 34d698f40bac 
Entire-Session: 76beb79f-6bc7-43c0-a87a-612aba368c74
Entire-Strategy: manual-commit
Entire-Agent: Claude Code
Ephemeral-branch: entire/937984d-e3b0c4
2026-03-29 11:13:43 +02:00

96 lines
16 KiB
Plaintext
Raw Permalink Blame History

Previously the throughput of the wal rotation was improved. But it is still not sufficient. It seems that on top of a continous metric instream there are also bursts of metric data. The problem occurs when a checkpoint is written: 3/29/2026, 8:06:02 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1440001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:01 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1430001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:01 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1420001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:01 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1410001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1400001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1390001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1380001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1370001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1360001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1350001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1340001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1330001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1320001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1310001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1300001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1290001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1280001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1270001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1260001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1250001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1240001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1230001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1220001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1210001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1200001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1190001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1180001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1170001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1160001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1150001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1140001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1130001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1120001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1110001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1100001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1090001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1080001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1070001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1060001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1050001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1040001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1030001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1020001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1010001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 1000001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 990001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 980001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 970001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 960001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 950001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 940001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 930001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 920001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 910001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 900001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 890001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 880001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 870001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 860001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 850001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 840001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 830001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 820001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 810001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 800001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 790001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 780001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 770001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 760001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 750001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 740001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 730001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 720001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 710001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 700001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 690001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:06:00 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 680001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 670001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 660001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 580001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 650001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 640001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 630001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 620001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 610001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 600001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 590001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 570001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 560001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 550001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 540001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 530001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 520001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 510001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:59 AMWARN[WARNING] lineprotocol.go:368: [METRICSTORE]> WAL shard channel full, dropped 500001 messages (data safe in memory, next snapshot will capture)
3/29/2026, 8:05:50 AMINFO[INFO] walCheckpoint.go:690: [METRICSTORE]> Starting binary checkpoint for 1845 hosts with 10 workers . Analyze the problem, improve the throughput and fix the issue.
Reference in New Issue View Git Blame Copy Permalink