Commit Graph

183 Commits

Author SHA1 Message Date
15231bc683 Cleanup and adapt to new structure 2023-08-17 12:34:30 +02:00
c7a04328d9 Fix schema and tests 2023-08-17 10:35:16 +02:00
87ce4f63d4 Refactor auth module
Separate parts
Add user repository
Add user schema
2023-08-17 10:29:00 +02:00
90bdfcfbb6 Add secured subrouter for REST API
Rename IP filter option
Add array helper in util
2023-08-14 14:33:05 +02:00
42e05fc999 Merge branch '189-refactor-authentication-module' into 105_modify_user_via_api 2023-08-14 14:00:27 +02:00
4a2afc7a5a Add LDAPSyncOnLogin option
Cleanup
Extend docs
Remove obsolete Expiration attribute
2023-08-14 12:40:21 +02:00
19d645f65c Readd URL token and cleanup
Fix session values.
2023-08-12 09:02:41 +02:00
Christoph Kluge
6393035e55 first iteraton of implementing ip-secured enpoint 2023-08-03 17:47:09 +02:00
6d8a3aa256 Fix bug in fsBackend and add tests for file utils 2023-06-28 07:39:39 +02:00
cf1b7e2db4 Fix typo in schema id 2023-06-27 15:35:29 +02:00
eb626db9c2 Debug compression service 2023-06-27 14:29:56 +02:00
d1e3e06b10 Add insert timestamp to tag, jobtag
Fixes #43
2023-06-27 09:41:06 +02:00
Jan Eitzinger
bf9b1c7d83
Merge pull request #147 from ClusterCockpit/111_fix_metric_names_and_stats
111 fix metric names and stats
2023-06-15 11:26:20 +02:00
9137931b74 Add cpu_load and mem_used to Job stats
Fixes #111
2023-06-15 11:07:48 +02:00
Christoph Kluge
7a40877c2c Merge branch 'hotfix' of https://github.com/ClusterCockpit/cc-backend into hotfix 2023-06-15 10:10:52 +02:00
Christoph Kluge
d49366d19c Switch from randomly chosen value to known value
- Case: Requesting Acc Metrics for running jobs
2023-06-15 10:10:49 +02:00
Jan Eitzinger
113cb863e0
Merge pull request #143 from ClusterCockpit/102-add-rest-api-endpoint-to-query-full-job-data-including-all-metric-timeseries
102 add rest api endpoint to query full job data including all metric timeseries
2023-06-15 07:06:45 +02:00
c662ced7e7 Initial rest endpoint for metrics data 2023-06-14 14:33:36 +02:00
93396aa0ea Fix bug in compression service 2023-06-14 07:31:43 +02:00
911dcb6626 Improve Compression Service
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
918f1993f2 Add loglevel access. Fix timer labels. 2023-06-09 15:02:22 +02:00
5ba6f0ed3a Refactor and adapt to new API 2023-06-09 09:09:41 +02:00
Christoph Kluge
a6cb833843 Merge branch 'master' into 97_107_mark_and_show_shared
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
65cebf6fde Add fix for broken ids in job archives 2023-06-01 15:26:53 +02:00
Christoph Kluge
37d5db443f Merge branch 'master' into 97_107_mark_and_show_shared 2023-06-01 15:24:26 +02:00
0a472d2b39 Fix bug in logger 2023-06-01 10:12:45 +02:00
Christoph Kluge
270750a78d Move concurrentJobs as field to main job query 2023-05-16 12:42:06 +02:00
0f9b83e636 Add Move retention policy
* Currently not testet!
2023-05-15 16:57:31 +02:00
1ae34c5e10 Extend archive manager 2023-05-15 14:32:23 +02:00
c64b996742 Finish uncompressed load data benchmark 2023-05-12 15:40:21 +02:00
c6dceb1265 Add LoadData Benchmark 2023-05-12 15:10:04 +02:00
6aea486891 Add util package 2023-05-12 15:09:39 +02:00
cfafd5aa08 Add archive test and fix fsBackend test 2023-05-11 16:17:17 +02:00
19e3ba7290 Improve retention
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
538427d59b Introduce Retention and compression 2023-05-09 16:33:26 +02:00
72f178a088 Extend config schema 2023-05-09 09:34:03 +02:00
d3107dd2cc Omit redundant init of handlers 2023-05-09 07:48:46 +02:00
07c10d88b9 Initialize log handlers also without init 2023-05-09 07:47:54 +02:00
3d3e8f1d5a Merge branch 'master' into 103-add-retention-support 2023-05-09 07:18:50 +02:00
0a43c4a146 Fix archive test
Add testdata
2023-05-04 18:54:34 +02:00
d9b18d1763 Cleanup
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
dce02ab1e0 Revert optional pointer attributes 2023-05-04 07:00:47 +02:00
5a09277d1c Restructure black box test
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25 Synchronize DB Schema with json schema
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
7c2171afda Save state 2023-04-18 07:43:21 +02:00
Jan Eitzinger
10f4e09562 Set all versions and print in version flag 2023-04-12 10:43:46 +02:00
Jan Eitzinger
ecd0709851 Fix more merge bugs 2023-04-12 07:30:41 +02:00
Jan Eitzinger
129dd13fc8 Fix merge errors 2023-04-11 16:26:09 +02:00
Jan Eitzinger
f8ba79e9e7 Merge branch 'master' into import-data-sanitation 2023-04-07 08:57:42 +02:00
Jan Eitzinger
52a07b51d2 Fix tests 2023-04-07 08:04:40 +02:00
Jan Eitzinger
af51c93601 Merge branch 'import-data-sanitation' of github.com:ClusterCockpit/cc-backend into import-data-sanitation 2023-04-06 18:09:39 +02:00
Jan Eitzinger
d858868901 Fine tune schema and job archive migration 2023-04-06 18:09:36 +02:00
Christoph Kluge
323066cb30 Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-04-06 11:38:12 +02:00
Christoph Kluge
f0cbb8c5dc Fix PrintList implementation, add NodeCount method 2023-04-06 11:38:10 +02:00
Jan Eitzinger
6661937fed Adapt schema and fix bugs in migration 2023-04-05 15:55:04 +02:00
Jan Eitzinger
559445900d Adapt job iterator
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
bfee9d847b Fix copy and paste error 2023-03-31 12:33:14 +02:00
Jan Eitzinger
e009caa3d9 Formatting. Fix Typo 2023-03-31 11:30:33 +02:00
Jan Eitzinger
eb2e2cb1d2 Implement tool to validate archive 2023-03-31 11:25:12 +02:00
Christoph Kluge
68a839bf1c Remove comments 2023-03-30 15:23:31 +02:00
Christoph Kluge
0c1b66aad9 Adapt svelte to new schema, add removed metric box
- Moved 'scope' field to parent jobMetric
- Implemented unit { prefix, base } where necessary
- SubCluster Metric Config 'remove' option implemented in Joblists
2023-03-30 15:21:35 +02:00
Christoph Kluge
1bb2827aeb Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-03-29 10:39:42 +02:00
Christoph Kluge
fc133f328a Add check to ccms getSubCluster routine
- Fixes nil pointer panic if topology requested from undef'd subcluster
- rest.go: vscode LINT
2023-03-29 10:39:31 +02:00
Jan Eitzinger
5d535edb48 Synchronize go job struct with schema 2023-03-29 06:46:33 +02:00
Jan Eitzinger
b5b5feb850 Fix tests 2023-03-27 14:41:00 +02:00
8447d011ad Add version to job archive 2023-03-27 13:24:06 +02:00
Jan Eitzinger
a4a90074ee Introduce compression in job archive. 2023-03-27 11:11:14 +02:00
Jan Eitzinger
58f589e019 Fix subcluster assigment with wildcard 2023-03-24 15:56:27 +01:00
Jan Eitzinger
3ce8344275 Add default match for one subcluster 2023-03-24 15:21:31 +01:00
Jan Eitzinger
1f8c6064e2 Refactor schema
* Remove redundant scope
* Change Series Id type to string
* Adopt golang structs to schema
2023-03-22 19:21:11 +01:00
Jan Eitzinger
a168efa8c8 Fix unit tests 2023-03-22 07:05:41 +01:00
5d526e1168 Adapt Cluster schema
Add remove boolean to subClusterConfig in MetricList.
Fix Linter errors.
2023-03-21 11:51:11 +01:00
efde2cbb8e Introduce hierarchical metriclists and thresholds 2023-03-18 08:15:11 +01:00
Jan Eitzinger
97f391deff GraphQL remove toplevel thresholds and add Unit
Add Unit to metric values in subCluster
2023-03-10 12:13:40 +01:00
Christoph Kluge
a651219a2c
Merge pull request #96 from ClusterCockpit/92_add_shortjobduration_config
Move ShortRunningJobsDuration value to root config
2023-03-09 16:43:43 +01:00
Christoph Kluge
4f1f034702 Move ShortRunningJobsDuration value to root config 2023-03-01 16:34:59 +01:00
9d28cb83f0 Remove swagger enum comment and rebuild Swagger docs
Relates to #95
2023-03-01 10:49:08 +01:00
c6c217adb0 Backup commit 2023-02-28 09:33:55 +01:00
3b38d8042e Fix tests 2023-02-15 13:07:19 +01:00
033598a656 Remove loglevel notice 2023-02-15 11:50:51 +01:00
Christoph Kluge
bb9593964a Reenable formatted log methods 2023-02-09 16:24:29 +01:00
Jan Eitzinger
96ad865b53 Cleanup and linting 2023-02-09 15:43:38 +01:00
Jan Eitzinger
c123a87ece Simplify log wrapper implementation 2023-02-09 14:14:58 +01:00
Christoph Kluge
a885e69125 Adapt loglevel for logs, shorten strings, fix formats, streamline
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5 Add log messages to error events w/o log message, primaryly error level
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
bd770d0e32 Fix return error, fix wrong log path, add notice log where applicable
- Adresses issue #26
2023-01-24 12:02:29 +01:00
Christoph Kluge
79a949b55e Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19 add more information to existing errors logs and panics
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Christoph Kluge
a39fc73345 Rerun swagger and fix typo 2022-11-30 12:40:07 +01:00
Jan Eitzinger
948da8f10b Review Sagger config. Delete Job endpoints 2022-11-11 15:26:27 +01:00
Jan Eitzinger
8dc9e6ecdd Fix issues in conversion tool 2022-11-10 13:37:53 +01:00
Jan Eitzinger
740d2a2928 Adapt test input to new unit schema 2022-11-09 19:47:56 +01:00
Jan Eitzinger
6200190763
Merge pull request #63 from giesselmann/fix_tag_disabled_repo
fix adding tag with disabled archive
2022-11-09 14:26:53 +01:00
Jan Eitzinger
dffd393c0d Reformatting 2022-11-09 14:05:27 +01:00
Pay Giesselmann
0de1368858 fix adding tag with disabled archive 2022-11-08 17:17:15 +01:00
Jan Eitzinger
9f9cc0c3ca Fix inconsistency in config
Fixes #60
2022-11-05 08:22:44 +01:00
Jan Eitzinger
74a8709edd Allow dash outside range of nodelist
Fixes #59
2022-11-04 07:31:36 +01:00
Michael Schwarz
f817ac5240 Accept externally generated JWTs provided via cookie
If there is an external service like an AuthAPI that can generate JWTs and
  hand them over to ClusterCockpit via cookies, CC can be configured to
  accept them
2022-10-19 13:36:13 +02:00
Jan Eitzinger
d2818d54ff Remove redundant stopTime property 2022-10-18 07:32:56 +02:00
Jan Eitzinger
29d215fcea Intermediate Save commit 2022-10-04 10:12:35 +02:00