Commit Graph

57 Commits

Author SHA1 Message Date
3bb5ae696a Add S3 loadJobMeta 2024-01-25 15:43:06 +01:00
578a270932 Start to port API 2023-11-09 06:19:53 +01:00
6d8a3aa256 Fix bug in fsBackend and add tests for file utils 2023-06-28 07:39:39 +02:00
eb626db9c2 Debug compression service 2023-06-27 14:29:56 +02:00
93396aa0ea Fix bug in compression service 2023-06-14 07:31:43 +02:00
911dcb6626 Improve Compression Service
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
0f9b83e636 Add Move retention policy
* Currently not testet!
2023-05-15 16:57:31 +02:00
1ae34c5e10 Extend archive manager 2023-05-15 14:32:23 +02:00
c64b996742 Finish uncompressed load data benchmark 2023-05-12 15:40:21 +02:00
c6dceb1265 Add LoadData Benchmark 2023-05-12 15:10:04 +02:00
6aea486891 Add util package 2023-05-12 15:09:39 +02:00
cfafd5aa08 Add archive test and fix fsBackend test 2023-05-11 16:17:17 +02:00
538427d59b Introduce Retention and compression 2023-05-09 16:33:26 +02:00
72f178a088 Extend config schema 2023-05-09 09:34:03 +02:00
3d3e8f1d5a Merge branch 'master' into 103-add-retention-support 2023-05-09 07:18:50 +02:00
0a43c4a146 Fix archive test
Add testdata
2023-05-04 18:54:34 +02:00
Jan Eitzinger
7c2171afda Save state 2023-04-18 07:43:21 +02:00
Jan Eitzinger
10f4e09562 Set all versions and print in version flag 2023-04-12 10:43:46 +02:00
Jan Eitzinger
ecd0709851 Fix more merge bugs 2023-04-12 07:30:41 +02:00
Jan Eitzinger
129dd13fc8 Fix merge errors 2023-04-11 16:26:09 +02:00
Jan Eitzinger
f8ba79e9e7 Merge branch 'master' into import-data-sanitation 2023-04-07 08:57:42 +02:00
Jan Eitzinger
af51c93601 Merge branch 'import-data-sanitation' of github.com:ClusterCockpit/cc-backend into import-data-sanitation 2023-04-06 18:09:39 +02:00
Jan Eitzinger
d858868901 Fine tune schema and job archive migration 2023-04-06 18:09:36 +02:00
Christoph Kluge
f0cbb8c5dc Fix PrintList implementation, add NodeCount method 2023-04-06 11:38:10 +02:00
Jan Eitzinger
559445900d Adapt job iterator
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
bfee9d847b Fix copy and paste error 2023-03-31 12:33:14 +02:00
Jan Eitzinger
eb2e2cb1d2 Implement tool to validate archive 2023-03-31 11:25:12 +02:00
Christoph Kluge
68a839bf1c Remove comments 2023-03-30 15:23:31 +02:00
Christoph Kluge
0c1b66aad9 Adapt svelte to new schema, add removed metric box
- Moved 'scope' field to parent jobMetric
- Implemented unit { prefix, base } where necessary
- SubCluster Metric Config 'remove' option implemented in Joblists
2023-03-30 15:21:35 +02:00
Christoph Kluge
fc133f328a Add check to ccms getSubCluster routine
- Fixes nil pointer panic if topology requested from undef'd subcluster
- rest.go: vscode LINT
2023-03-29 10:39:31 +02:00
Jan Eitzinger
b5b5feb850 Fix tests 2023-03-27 14:41:00 +02:00
8447d011ad Add version to job archive 2023-03-27 13:24:06 +02:00
Jan Eitzinger
a4a90074ee Introduce compression in job archive. 2023-03-27 11:11:14 +02:00
Jan Eitzinger
58f589e019 Fix subcluster assigment with wildcard 2023-03-24 15:56:27 +01:00
Jan Eitzinger
3ce8344275 Add default match for one subcluster 2023-03-24 15:21:31 +01:00
Jan Eitzinger
a168efa8c8 Fix unit tests 2023-03-22 07:05:41 +01:00
3b38d8042e Fix tests 2023-02-15 13:07:19 +01:00
Christoph Kluge
a885e69125 Adapt loglevel for logs, shorten strings, fix formats, streamline
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5 Add log messages to error events w/o log message, primaryly error level
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
79a949b55e Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19 add more information to existing errors logs and panics
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Jan Eitzinger
6200190763
Merge pull request #63 from giesselmann/fix_tag_disabled_repo
fix adding tag with disabled archive
2022-11-09 14:26:53 +01:00
Jan Eitzinger
dffd393c0d Reformatting 2022-11-09 14:05:27 +01:00
Pay Giesselmann
0de1368858 fix adding tag with disabled archive 2022-11-08 17:17:15 +01:00
Jan Eitzinger
74a8709edd Allow dash outside range of nodelist
Fixes #59
2022-11-04 07:31:36 +01:00
Jan Eitzinger
5c715b6ec2 Add test for import job handler. Improve error messages. 2022-09-23 15:23:45 +02:00
Jan Eitzinger
35114aeb2d Cleanup. Fix failing test 2022-09-20 13:07:46 +02:00
Jan Eitzinger
777f225e91 Add option for json validation 2022-09-13 15:22:20 +02:00
Jan Eitzinger
520c814e3b Introduce import job flag 2022-09-13 15:20:07 +02:00
Jan Eitzinger
fd100f28dd Change interface and reintroduce caching 2022-09-13 07:37:48 +02:00