Commit Graph

165 Commits

Author SHA1 Message Date
93396aa0ea Fix bug in compression service 2023-06-14 07:31:43 +02:00
911dcb6626 Improve Compression Service
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
918f1993f2 Add loglevel access. Fix timer labels. 2023-06-09 15:02:22 +02:00
5ba6f0ed3a Refactor and adapt to new API 2023-06-09 09:09:41 +02:00
Christoph Kluge
a6cb833843 Merge branch 'master' into 97_107_mark_and_show_shared
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
65cebf6fde Add fix for broken ids in job archives 2023-06-01 15:26:53 +02:00
Christoph Kluge
37d5db443f Merge branch 'master' into 97_107_mark_and_show_shared 2023-06-01 15:24:26 +02:00
0a472d2b39 Fix bug in logger 2023-06-01 10:12:45 +02:00
Christoph Kluge
270750a78d Move concurrentJobs as field to main job query 2023-05-16 12:42:06 +02:00
0f9b83e636 Add Move retention policy
* Currently not testet!
2023-05-15 16:57:31 +02:00
1ae34c5e10 Extend archive manager 2023-05-15 14:32:23 +02:00
c64b996742 Finish uncompressed load data benchmark 2023-05-12 15:40:21 +02:00
c6dceb1265 Add LoadData Benchmark 2023-05-12 15:10:04 +02:00
6aea486891 Add util package 2023-05-12 15:09:39 +02:00
cfafd5aa08 Add archive test and fix fsBackend test 2023-05-11 16:17:17 +02:00
19e3ba7290 Improve retention
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
538427d59b Introduce Retention and compression 2023-05-09 16:33:26 +02:00
72f178a088 Extend config schema 2023-05-09 09:34:03 +02:00
d3107dd2cc Omit redundant init of handlers 2023-05-09 07:48:46 +02:00
07c10d88b9 Initialize log handlers also without init 2023-05-09 07:47:54 +02:00
3d3e8f1d5a Merge branch 'master' into 103-add-retention-support 2023-05-09 07:18:50 +02:00
0a43c4a146 Fix archive test
Add testdata
2023-05-04 18:54:34 +02:00
d9b18d1763 Cleanup
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
dce02ab1e0 Revert optional pointer attributes 2023-05-04 07:00:47 +02:00
5a09277d1c Restructure black box test
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25 Synchronize DB Schema with json schema
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
7c2171afda Save state 2023-04-18 07:43:21 +02:00
Jan Eitzinger
10f4e09562 Set all versions and print in version flag 2023-04-12 10:43:46 +02:00
Jan Eitzinger
ecd0709851 Fix more merge bugs 2023-04-12 07:30:41 +02:00
Jan Eitzinger
129dd13fc8 Fix merge errors 2023-04-11 16:26:09 +02:00
Jan Eitzinger
f8ba79e9e7 Merge branch 'master' into import-data-sanitation 2023-04-07 08:57:42 +02:00
Jan Eitzinger
52a07b51d2 Fix tests 2023-04-07 08:04:40 +02:00
Jan Eitzinger
af51c93601 Merge branch 'import-data-sanitation' of github.com:ClusterCockpit/cc-backend into import-data-sanitation 2023-04-06 18:09:39 +02:00
Jan Eitzinger
d858868901 Fine tune schema and job archive migration 2023-04-06 18:09:36 +02:00
Christoph Kluge
323066cb30 Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-04-06 11:38:12 +02:00
Christoph Kluge
f0cbb8c5dc Fix PrintList implementation, add NodeCount method 2023-04-06 11:38:10 +02:00
Jan Eitzinger
6661937fed Adapt schema and fix bugs in migration 2023-04-05 15:55:04 +02:00
Jan Eitzinger
559445900d Adapt job iterator
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
bfee9d847b Fix copy and paste error 2023-03-31 12:33:14 +02:00
Jan Eitzinger
e009caa3d9 Formatting. Fix Typo 2023-03-31 11:30:33 +02:00
Jan Eitzinger
eb2e2cb1d2 Implement tool to validate archive 2023-03-31 11:25:12 +02:00
Christoph Kluge
68a839bf1c Remove comments 2023-03-30 15:23:31 +02:00
Christoph Kluge
0c1b66aad9 Adapt svelte to new schema, add removed metric box
- Moved 'scope' field to parent jobMetric
- Implemented unit { prefix, base } where necessary
- SubCluster Metric Config 'remove' option implemented in Joblists
2023-03-30 15:21:35 +02:00
Christoph Kluge
1bb2827aeb Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-03-29 10:39:42 +02:00
Christoph Kluge
fc133f328a Add check to ccms getSubCluster routine
- Fixes nil pointer panic if topology requested from undef'd subcluster
- rest.go: vscode LINT
2023-03-29 10:39:31 +02:00
Jan Eitzinger
5d535edb48 Synchronize go job struct with schema 2023-03-29 06:46:33 +02:00
Jan Eitzinger
b5b5feb850 Fix tests 2023-03-27 14:41:00 +02:00
8447d011ad Add version to job archive 2023-03-27 13:24:06 +02:00
Jan Eitzinger
a4a90074ee Introduce compression in job archive. 2023-03-27 11:11:14 +02:00
Jan Eitzinger
58f589e019 Fix subcluster assigment with wildcard 2023-03-24 15:56:27 +01:00
Jan Eitzinger
3ce8344275 Add default match for one subcluster 2023-03-24 15:21:31 +01:00
Jan Eitzinger
1f8c6064e2 Refactor schema
* Remove redundant scope
* Change Series Id type to string
* Adopt golang structs to schema
2023-03-22 19:21:11 +01:00
Jan Eitzinger
a168efa8c8 Fix unit tests 2023-03-22 07:05:41 +01:00
5d526e1168 Adapt Cluster schema
Add remove boolean to subClusterConfig in MetricList.
Fix Linter errors.
2023-03-21 11:51:11 +01:00
efde2cbb8e Introduce hierarchical metriclists and thresholds 2023-03-18 08:15:11 +01:00
Jan Eitzinger
97f391deff GraphQL remove toplevel thresholds and add Unit
Add Unit to metric values in subCluster
2023-03-10 12:13:40 +01:00
Christoph Kluge
a651219a2c
Merge pull request #96 from ClusterCockpit/92_add_shortjobduration_config
Move ShortRunningJobsDuration value to root config
2023-03-09 16:43:43 +01:00
Christoph Kluge
4f1f034702 Move ShortRunningJobsDuration value to root config 2023-03-01 16:34:59 +01:00
9d28cb83f0 Remove swagger enum comment and rebuild Swagger docs
Relates to #95
2023-03-01 10:49:08 +01:00
c6c217adb0 Backup commit 2023-02-28 09:33:55 +01:00
3b38d8042e Fix tests 2023-02-15 13:07:19 +01:00
033598a656 Remove loglevel notice 2023-02-15 11:50:51 +01:00
Christoph Kluge
bb9593964a Reenable formatted log methods 2023-02-09 16:24:29 +01:00
Jan Eitzinger
96ad865b53 Cleanup and linting 2023-02-09 15:43:38 +01:00
Jan Eitzinger
c123a87ece Simplify log wrapper implementation 2023-02-09 14:14:58 +01:00
Christoph Kluge
a885e69125 Adapt loglevel for logs, shorten strings, fix formats, streamline
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5 Add log messages to error events w/o log message, primaryly error level
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
bd770d0e32 Fix return error, fix wrong log path, add notice log where applicable
- Adresses issue #26
2023-01-24 12:02:29 +01:00
Christoph Kluge
79a949b55e Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19 add more information to existing errors logs and panics
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Christoph Kluge
a39fc73345 Rerun swagger and fix typo 2022-11-30 12:40:07 +01:00
Jan Eitzinger
948da8f10b Review Sagger config. Delete Job endpoints 2022-11-11 15:26:27 +01:00
Jan Eitzinger
8dc9e6ecdd Fix issues in conversion tool 2022-11-10 13:37:53 +01:00
Jan Eitzinger
740d2a2928 Adapt test input to new unit schema 2022-11-09 19:47:56 +01:00
Jan Eitzinger
6200190763
Merge pull request #63 from giesselmann/fix_tag_disabled_repo
fix adding tag with disabled archive
2022-11-09 14:26:53 +01:00
Jan Eitzinger
dffd393c0d Reformatting 2022-11-09 14:05:27 +01:00
Pay Giesselmann
0de1368858 fix adding tag with disabled archive 2022-11-08 17:17:15 +01:00
Jan Eitzinger
9f9cc0c3ca Fix inconsistency in config
Fixes #60
2022-11-05 08:22:44 +01:00
Jan Eitzinger
74a8709edd Allow dash outside range of nodelist
Fixes #59
2022-11-04 07:31:36 +01:00
Michael Schwarz
f817ac5240 Accept externally generated JWTs provided via cookie
If there is an external service like an AuthAPI that can generate JWTs and
  hand them over to ClusterCockpit via cookies, CC can be configured to
  accept them
2022-10-19 13:36:13 +02:00
Jan Eitzinger
d2818d54ff Remove redundant stopTime property 2022-10-18 07:32:56 +02:00
Jan Eitzinger
29d215fcea Intermediate Save commit 2022-10-04 10:12:35 +02:00
Jan Eitzinger
7251344d4a Add convenience routines to unit package 2022-09-28 16:25:26 +02:00
Jan Eitzinger
918a07735d
Merge branch 'master' into dev-job-archive-module 2022-09-26 15:12:46 +02:00
Jan Eitzinger
5c715b6ec2 Add test for import job handler. Improve error messages. 2022-09-23 15:23:45 +02:00
Jan Eitzinger
3a582ee2f5 Update schema 2022-09-21 15:20:26 +02:00
Jan Eitzinger
4652f1dae1 Extend unit schema 2022-09-21 15:10:31 +02:00
Christoph Kluge
6712b15034 Better represent 'exclusiveMinimum' from JSON schemata 2022-09-21 14:21:13 +02:00
Christoph Kluge
94c77d8a67 Small additions to swagger docs 2022-09-21 14:12:24 +02:00
Christoph Kluge
05a84cb95c Extend parsed swagger documentation by adding comments 2022-09-21 11:54:19 +02:00
Jan Eitzinger
35114aeb2d Cleanup. Fix failing test 2022-09-20 13:07:46 +02:00
Jan Eitzinger
0c4096eda3 Fix api testcase 2022-09-20 12:21:20 +02:00
Jan Eitzinger
2fdf1c14be Fix broken schema and add tests 2022-09-20 10:27:23 +02:00
Jan Eitzinger
207254a9e2 Embed schema files for validation 2022-09-20 07:05:01 +02:00
Jan Eitzinger
e8d7722c42 Add json schema validation for config file 2022-09-19 16:16:05 +02:00
Jan Eitzinger
8845bba90d Fix bug 2022-09-13 15:40:32 +02:00
Jan Eitzinger
777f225e91 Add option for json validation 2022-09-13 15:22:20 +02:00
Jan Eitzinger
520c814e3b Introduce import job flag 2022-09-13 15:20:07 +02:00
Jan Eitzinger
f6508046ff Integrate cc-units as units package 2022-09-13 07:38:44 +02:00
Jan Eitzinger
d8aaa26c27 Cleanup and small fixes 2022-09-13 07:38:18 +02:00