2024-06-28 07:31:41 +02:00
|
|
|
# Possible Tasks and Improvements
|
2021-06-09 06:03:31 +02:00
|
|
|
|
2024-06-28 07:31:41 +02:00
|
|
|
- Benchmarking
|
|
|
|
|
- Benchmark and compare common timeseries DBs with our data and our queries
|
|
|
|
|
- Memory management
|
|
|
|
|
- To overcome garbage collection overhead: Reimplement in Rust
|
|
|
|
|
- Request memory directly batchwise via mmap (started in branch)
|
|
|
|
|
- Archive
|
|
|
|
|
- S3 backend for archive
|
2024-06-18 07:27:27 +02:00
|
|
|
- Store information in each buffer if already archived
|
|
|
|
|
- Do not create new checkpoint if all buffers already archived
|
2024-06-28 07:31:41 +02:00
|
|
|
- Checkpoints
|
|
|
|
|
- S3 backend for checkpoints
|
|
|
|
|
- Combine checkpoints into larger files
|
|
|
|
|
- Binary checkpoints (started in branch)
|
|
|
|
|
- API
|
|
|
|
|
- Redesign query interface
|
|
|
|
|
- Introduce JWT authentication for REST and NATS
|
|
|
|
|
- Testing
|
2024-06-18 07:27:27 +02:00
|
|
|
- General tests
|
|
|
|
|
- Check for corner cases that should fail gracefully
|
2024-06-28 07:31:41 +02:00
|
|
|
- Write a more realistic `ToArchive`/`FromArchive` Tests
|
|
|
|
|
- Aggregation
|
|
|
|
|
- Calculate averages buffer-wise as soon as full, average weighted by length of buffer
|
2024-06-18 07:27:27 +02:00
|
|
|
- Only the head-buffer needs to be fully traversed
|
2024-06-28 07:31:41 +02:00
|
|
|
- If aggregating over hwthreads/cores/sockets cache those results and reuse
|
|
|
|
|
some of that for new queries aggregating only over the newer data
|
|
|
|
|
- Compression
|
|
|
|
|
- Enable compression for http API requests
|
|
|
|
|
- Enable compression for checkpoints/archive
|
|
|
|
|
- Sampling
|
|
|
|
|
- Support data re sampling to reduce data points
|
|
|
|
|
- Support re sampling algorithms that preserve min/max as far as possible
|
