mirror of
https://github.com/ClusterCockpit/cc-metric-store.git
synced 2024-11-10 05:07:25 +01:00
34 lines
1.3 KiB
Markdown
34 lines
1.3 KiB
Markdown
# Possible Tasks and Improvements
|
|
|
|
- Benchmarking
|
|
- Benchmark and compare common timeseries DBs with our data and our queries
|
|
- Memory management
|
|
- To overcome garbage collection overhead: Reimplement in Rust
|
|
- Request memory directly batchwise via mmap (started in branch)
|
|
- Archive
|
|
- S3 backend for archive
|
|
- Store information in each buffer if already archived
|
|
- Do not create new checkpoint if all buffers already archived
|
|
- Checkpoints
|
|
- S3 backend for checkpoints
|
|
- Combine checkpoints into larger files
|
|
- Binary checkpoints (started in branch)
|
|
- API
|
|
- Redesign query interface
|
|
- Introduce JWT authentication for REST and NATS
|
|
- Testing
|
|
- General tests
|
|
- Check for corner cases that should fail gracefully
|
|
- Write a more realistic `ToArchive`/`FromArchive` Tests
|
|
- Aggregation
|
|
- Calculate averages buffer-wise as soon as full, average weighted by length of buffer
|
|
- Only the head-buffer needs to be fully traversed
|
|
- If aggregating over hwthreads/cores/sockets cache those results and reuse
|
|
some of that for new queries aggregating only over the newer data
|
|
- Compression
|
|
- Enable compression for http API requests
|
|
- Enable compression for checkpoints/archive
|
|
- Sampling
|
|
- Support data re sampling to reduce data points
|
|
- Support re sampling algorithms that preserve min/max as far as possible
|