Commit Graph

453 Commits

Author SHA1 Message Date
Christoph Kluge
6a1e35107f fix: analysis metric histogram normalized by scope
- native acc metrics normalized by accHours
- native core metrics normalized by coreHours
2023-08-24 11:52:36 +02:00
Jan Eitzinger
b7aacd1b33
Merge pull request #194 from ClusterCockpit/105_modify_user_via_api
first iteraton of implementing ip-secured enpoint
2023-08-23 08:57:40 +02:00
Christoph Kluge
2f35482aff Merge branch 'master' into 166_add_scopes_analysis 2023-08-22 15:26:20 +02:00
Christoph Kluge
f36f62fb47 Improve user endpoint swagger docs 2023-08-21 12:12:28 +02:00
dc67a1f103 Update frontend and backend dependencies 2023-08-21 08:49:42 +02:00
Christoph Kluge
f6c4c963ec feat: Add users rest endpoint swagger docs 2023-08-18 17:18:31 +02:00
ebcae32e23 Update docs and cleanup 2023-08-18 15:56:11 +02:00
cda46141cc Cleanup and add wildcard for IP Filter 2023-08-18 13:03:11 +02:00
3028f60807 Reformat and add debug output 2023-08-18 11:59:16 +02:00
Christoph Kluge
734e818b19 Merge branch '189-refactor-authentication-module' of https://github.com/ClusterCockpit/cc-backend into 189-refactor-authentication-module 2023-08-18 11:17:33 +02:00
Christoph Kluge
57bda63506 Cleanup some error strings 2023-08-18 11:17:31 +02:00
da551a0bb4 Repair broken error handlng 2023-08-18 11:00:13 +02:00
32b0c8bdd7 Refactor and cleanup Auth configuration 2023-08-18 10:43:06 +02:00
14c487c9e4 Update test inputs 2023-08-18 09:31:57 +02:00
56d559fdd7 Fix bug with jwt max-age option 2023-08-18 09:19:30 +02:00
cfcf939339 Add config to jwt again 2023-08-18 08:57:56 +02:00
d51be5c308 Formatting and minor fixes 2023-08-18 08:49:25 +02:00
29552fadc3 Cleanup SyncOnLogin Handling 2023-08-17 14:02:04 +02:00
15231bc683 Cleanup and adapt to new structure 2023-08-17 12:34:30 +02:00
87ce4f63d4 Refactor auth module
Separate parts
Add user repository
Add user schema
2023-08-17 10:29:00 +02:00
80aed87415 Retry fetching user after CanLogin 2023-08-16 17:21:12 +02:00
65cf86586a Merge branch '105_modify_user_via_api' into 189-refactor-authentication-module 2023-08-16 09:46:41 +02:00
4f6d1fec68 Fix errors in ldap auth 2023-08-16 09:19:41 +02:00
202521cbfd Restructure routing and security check 2023-08-14 18:38:30 +02:00
90bdfcfbb6 Add secured subrouter for REST API
Rename IP filter option
Add array helper in util
2023-08-14 14:33:05 +02:00
42e05fc999 Merge branch '189-refactor-authentication-module' into 105_modify_user_via_api 2023-08-14 14:00:27 +02:00
Christoph Kluge
fe6de5bc68 Merge branch '189-refactor-authentication-module' of https://github.com/ClusterCockpit/cc-backend into 189-refactor-authentication-module 2023-08-14 13:52:29 +02:00
Christoph Kluge
e550e57ac0 Fix Java/Grails issued token parsing
- Tested locally until successfull login
- Initialize empty projects array
2023-08-14 13:52:26 +02:00
4a2afc7a5a Add LDAPSyncOnLogin option
Cleanup
Extend docs
Remove obsolete Expiration attribute
2023-08-14 12:40:21 +02:00
19d645f65c Readd URL token and cleanup
Fix session values.
2023-08-12 09:02:41 +02:00
b8273a9b02 refactor auth module
Restructure module
Separate JWT auth variants
Cleanup code
Fixes #189
2023-08-11 10:00:23 +02:00
Pay Giesselmann
d929bdc9a1 omit metrics with empty series 2023-08-08 10:19:37 +02:00
Christoph Kluge
6393035e55 first iteraton of implementing ip-secured enpoint 2023-08-03 17:47:09 +02:00
Christoph Kluge
f7571211fd initial branch commit 2023-07-21 16:33:53 +02:00
Jan Eitzinger
c0ab5de2f1
Merge pull request #182 from ClusterCockpit/179_fix_frontend_apiusers
Fix frontend render for users with api role
2023-07-20 07:42:15 +02:00
Joachim Meyer
650bcae6be Add arrayJobId searchbar option. 2023-07-19 09:46:48 +02:00
Jan Eitzinger
536a51b93f
Merge pull request #186 from ClusterCockpit/185-add-notification-banner
185 add notification banner
2023-07-19 09:13:15 +02:00
04e8279ae4 Change log level for JWT Cross login warning to debug 2023-07-19 09:04:27 +02:00
c6a0d442cc feat: Add optional notification banner on homepage
Fixes #185
2023-07-19 08:25:14 +02:00
Christoph Kluge
55943cacbf Fix frontend render for users with api role 2023-07-17 12:19:49 +02:00
df9fd77d06 Refactor auth and add docs
Cleanup and reformat
2023-07-05 09:50:44 +02:00
Christoph Kluge
2f471dc192 Change listQuery to jobId array, adapt filter pipe 2023-06-30 16:55:34 +02:00
Christoph Kluge
c04aea89c9 Implement node filter in frontend, fix backend
- Add running job count and link to list to single node view
2023-06-30 12:01:27 +02:00
b5a7249ad5 Fix sqlite migration for configuration 2023-06-30 09:15:58 +02:00
dd80d5af9e Change config value datatype
Fixes #167
2023-06-29 14:55:38 +02:00
7174f27a89 Add node filter and concurrent job list query 2023-06-28 13:35:41 +02:00
3828c138b8 Add init flag 2023-06-28 12:41:27 +02:00
6d8a3aa256 Fix bug in fsBackend and add tests for file utils 2023-06-28 07:39:39 +02:00
34050c8ce0 Fix argument dashes 2023-06-27 14:30:09 +02:00
eb626db9c2 Debug compression service 2023-06-27 14:29:56 +02:00
d1e3e06b10 Add insert timestamp to tag, jobtag
Fixes #43
2023-06-27 09:41:06 +02:00
8aaa5722e9 Fix errors in query logic 2023-06-26 16:48:13 +02:00
99a2888431 Fix typo in Query 2023-06-26 14:45:49 +02:00
Jan Eitzinger
7d85c0e9ad
Merge pull request #162 from ClusterCockpit/refactor-concurrent-job-resolver
Refactor concurrent job resolver
2023-06-26 14:28:00 +02:00
Christoph Kluge
f992ffc2da Fix SSL timing issues with best guess search 2023-06-23 16:09:33 +02:00
Jan Eitzinger
2f6c16a7d8
Merge pull request #156 from giesselmann/log_levels
lower log level for frequent messages
2023-06-23 10:33:44 +02:00
Jan Eitzinger
76d3018b5c
Merge pull request #159 from ClusterCockpit/158_fix_searchbar
158 fix searchbar
2023-06-23 10:31:19 +02:00
Christoph Kluge
3c26deda30 Format message.tmpl 2023-06-23 09:42:31 +02:00
Christoph Kluge
9afb6dc933 Add Header Requirements and BuildInfos to message 2023-06-22 19:56:21 +02:00
8eda4b306d Unify and cleanup message template 2023-06-22 18:09:40 +02:00
82b8e8c284 Fix bug in SearchBar Handler
Introduce Message boxes
Incomplete and needs cleanup
2023-06-22 16:26:09 +02:00
Christoph Kluge
bcadb1adda Remove errorcase from single searchterm logic 2023-06-22 10:58:36 +02:00
feba722a53 Refactor and Cleanup
Add SecurityCheck
2023-06-22 06:26:19 +02:00
bb7c1005c9 Refactor query logic and move to job.go 2023-06-20 15:52:16 +02:00
Pay Giesselmann
a9544f5609 lower log level for frequent messages 2023-06-20 15:47:38 +02:00
Jan Eitzinger
d0516f12b0
Merge pull request #154 from ClusterCockpit/hotfix
Hotfix
2023-06-20 13:00:00 +02:00
c973a29734 Restore api role access in SecurityCheck 2023-06-20 12:54:26 +02:00
33613cdda0 Sync commit 2023-06-20 10:38:53 +02:00
Jan Eitzinger
07f8950838
Merge pull request #148 from ClusterCockpit/hotfix
Hotfix
2023-06-16 14:37:10 +02:00
f0685919fd Streamline auth error handling 2023-06-15 12:00:45 +02:00
17113eddca Fix failed test due to config change 2023-06-15 11:23:50 +02:00
9137931b74 Add cpu_load and mem_used to Job stats
Fixes #111
2023-06-15 11:07:48 +02:00
85c7b9e357 Add correct versions 2023-06-15 09:55:24 +02:00
Jan Eitzinger
113cb863e0
Merge pull request #143 from ClusterCockpit/102-add-rest-api-endpoint-to-query-full-job-data-including-all-metric-timeseries
102 add rest api endpoint to query full job data including all metric timeseries
2023-06-15 07:06:45 +02:00
54aa940d3e Add job metricdata rest endpoint
Fixes #102
2023-06-14 15:03:01 +02:00
e6a5874999 Fix bug if local login provides wrong pw
Fixes #140
2023-06-14 14:35:25 +02:00
c662ced7e7 Initial rest endpoint for metrics data 2023-06-14 14:33:36 +02:00
3766121aef Fix test 2023-06-13 14:40:23 +02:00
Christoph Kluge
f31d8a3af0 fix test 2023-06-12 11:58:10 +02:00
Christoph Kluge
0d2e20e9e4 Handle users with no roles as "user role"
-for backwards compatibility
2023-06-12 11:35:16 +02:00
911dcb6626 Improve Compression Service
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
Jan Eitzinger
a88a97f1b8 Fix bug for histogram only query 2023-06-09 16:03:06 +02:00
918f1993f2 Add loglevel access. Fix timer labels. 2023-06-09 15:02:22 +02:00
Christoph Kluge
edb1b47281 Add AddJobCount() for analysis view 2023-06-09 13:15:25 +02:00
616095fe66 Add additional job stats, fix test 2023-06-09 11:29:07 +02:00
5ba6f0ed3a Refactor and adapt to new API 2023-06-09 09:09:41 +02:00
02752f52e4 Merge branch 'master' into refactor-job-repository 2023-06-08 06:22:51 +02:00
fb86ebdbcc Renaming 2023-06-08 06:18:19 +02:00
Jan Eitzinger
e244a7c85a
Merge pull request #122 from pc2/master
Fix database migration order
2023-06-07 17:26:58 +02:00
Jan Eitzinger
1b1e46bf01
Merge pull request #123 from ClusterCockpit/121_fix_filter_reactivity_crash
121 fix filter reactivity crash
2023-06-07 17:24:46 +02:00
Michael Schwarz
a183c23e25 Fix truncate call on mysql and provide step 4 migrations for mysql 2023-06-07 16:49:08 +02:00
Christoph Kluge
4e568e60cd Del JobName from Job type, jobStats name required
- Regenerated GQL
- Relates to #121
- return jobStats name as string
2023-06-07 14:13:59 +02:00
ff23f8e270 Merge branch 'refactor-job-repository' of github.com:ClusterCockpit/cc-backend into refactor-job-repository 2023-06-07 12:00:11 +02:00
becb2bfa3a Refactor Jobs stats resolver 2023-06-07 11:58:58 +02:00
Michael Schwarz
df240c2394 Fix database migration order 2023-06-07 11:45:58 +02:00
784ae125be Reformatting 2023-06-06 13:13:08 +02:00
695e8e270a Split job Repository in multiple files 2023-06-06 10:27:55 +02:00
Jan Eitzinger
797d5da80b
Merge pull request #120 from ClusterCockpit/optimize-sqlite-db-setup
Enable SQL hooks for statement debugging
2023-06-05 16:41:52 +02:00
96d036a128 Enable SQL hooks for statement debugging 2023-06-05 16:38:58 +02:00
Jan Eitzinger
e32872181f
Merge pull request #118 from ClusterCockpit/optimize-sqlite-db-setup
Add migration for job table contraints
2023-06-05 16:13:24 +02:00
d53ef526dd Fix tests 2023-06-05 16:10:47 +02:00
e444b83149 Fix errors 2023-06-05 16:07:01 +02:00
5ca4bdbd91 Remove VACUUM 2023-06-05 16:01:14 +02:00
8b6a1c8045 Add migration for job table contraints 2023-06-05 15:58:07 +02:00
Christoph Kluge
a6cb833843 Merge branch 'master' into 97_107_mark_and_show_shared
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
2de85b0120 Add composite indexes 2023-06-01 15:40:37 +02:00
203ccdf73b Rename test db 2023-06-01 15:29:25 +02:00
447fc574e8 Add query benchmark 2023-06-01 15:27:19 +02:00
Christoph Kluge
37d5db443f Merge branch 'master' into 97_107_mark_and_show_shared 2023-06-01 15:24:26 +02:00
205a400c6f Also Benchmarks need _test in name 2023-06-01 14:10:10 +02:00
012b131b23 Refactor for benchmarking 2023-06-01 13:57:35 +02:00
0f87f651f9 Refactor DB connection settings 2023-06-01 08:03:12 +02:00
Christoph Kluge
5bb5e5ed24 Fix test: Change jobId conversion to fmt 2023-05-16 16:06:04 +02:00
Christoph Kluge
270750a78d Move concurrentJobs as field to main job query 2023-05-16 12:42:06 +02:00
1ae34c5e10 Extend archive manager 2023-05-15 14:32:23 +02:00
5beb84b575 Add copyright to util package 2023-05-12 15:40:33 +02:00
6aea486891 Add util package 2023-05-12 15:09:39 +02:00
19e3ba7290 Improve retention
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
cc634dd155 Improve build and error handling 2023-05-11 09:39:23 +02:00
538427d59b Introduce Retention and compression 2023-05-09 16:33:26 +02:00
788d2cd61c Fix repo tests 2023-05-05 06:02:17 +02:00
f6c629c696 Remove obsolete check
Now handeled by DB UNIQUE constraint
2023-05-04 18:14:46 +02:00
7491ae9e3c Fix typo in DB schema 2023-05-04 17:26:33 +02:00
066c4553b4 Refactor 2023-05-04 16:27:30 +02:00
5404be3ca7 Add unique constrain on job table 2023-05-04 16:04:57 +02:00
c5b67d64d2 Add Table flush on init 2023-05-04 16:03:04 +02:00
d9b18d1763 Cleanup
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
b878b31de5 Add test result files 2023-05-04 13:32:11 +02:00
23802abc10 Rebuilt GQL code 2023-05-04 13:22:46 +02:00
27894ea0a9 Restructure tests 2023-05-04 07:00:30 +02:00
Christoph Kluge
d93610f700 List parallel jobs on node for jobs on shared node
- Relates to issue #97
- required GQL schema extension and regeneration
- Works for archived jobs aswell
2023-04-28 12:34:40 +02:00
ebba4371eb Extract importer package and restructure tests 2023-04-28 08:49:58 +02:00
c090b18628 Sync commit 2023-04-26 14:08:39 +02:00
9299f65993 Start to integrate file based testing 2023-04-25 09:27:22 +02:00
5a09277d1c Restructure black box test
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25 Synchronize DB Schema with json schema
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
10f4e09562 Set all versions and print in version flag 2023-04-12 10:43:46 +02:00
Jan Eitzinger
129dd13fc8 Fix merge errors 2023-04-11 16:26:09 +02:00
Jan Eitzinger
f8ba79e9e7 Merge branch 'master' into import-data-sanitation 2023-04-07 08:57:42 +02:00
Jan Eitzinger
d420b8b666
Merge branch 'master' into 40_45_82_update_roles 2023-04-07 08:19:04 +02:00
Christoph Kluge
f0cbb8c5dc Fix PrintList implementation, add NodeCount method 2023-04-06 11:38:10 +02:00
Christoph Kluge
2401a2b940 Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-03-31 17:18:49 +02:00
Christoph Kluge
adc1d94e3f subcluster in metricSelect, add infobox to systems
- Default: Show Clusters next to Metrics
- New: If Cluster filter activem show subclusters instead (reactive)
- add note to analysis view
- systems view now with info boxes if metric is removed for subcluster
2023-03-31 17:18:16 +02:00
Jan Eitzinger
559445900d Adapt job iterator
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Christoph Kluge
68a839bf1c Remove comments 2023-03-30 15:23:31 +02:00
Christoph Kluge
0c1b66aad9 Adapt svelte to new schema, add removed metric box
- Moved 'scope' field to parent jobMetric
- Implemented unit { prefix, base } where necessary
- SubCluster Metric Config 'remove' option implemented in Joblists
2023-03-30 15:21:35 +02:00
Jan Eitzinger
82120cfb3d Update generated.go 2023-03-29 10:54:16 +02:00
Christoph Kluge
1bb2827aeb Merge branch 'import-data-sanitation' of https://github.com/ClusterCockpit/cc-backend into import-data-sanitation 2023-03-29 10:39:42 +02:00
Christoph Kluge
fc133f328a Add check to ccms getSubCluster routine
- Fixes nil pointer panic if topology requested from undef'd subcluster
- rest.go: vscode LINT
2023-03-29 10:39:31 +02:00
Jan Eitzinger
5d535edb48 Synchronize go job struct with schema 2023-03-29 06:46:33 +02:00
Jan Eitzinger
9aff980a38 Add scope to Metric List. Add numberOfNodes. 2023-03-24 15:10:23 +01:00
Jan Eitzinger
4d49698e5a Add scope to GraphQL Metric Type 2023-03-24 14:43:37 +01:00
Jan Eitzinger
1f8c6064e2 Refactor schema
* Remove redundant scope
* Change Series Id type to string
* Adopt golang structs to schema
2023-03-22 19:21:11 +01:00
Christoph Kluge
de71fee61e
Merge pull request #98 from giesselmann/fix-updateconfig
Fix UpdateConfig
2023-03-21 12:09:40 +01:00
5d526e1168 Adapt Cluster schema
Add remove boolean to subClusterConfig in MetricList.
Fix Linter errors.
2023-03-21 11:51:11 +01:00
747f6d4e2d Add MetricConfig in SubClusters. 2023-03-20 16:27:36 +01:00
Pay Giesselmann
de822e13fd
Fix UpdateConfig 2023-03-13 10:45:55 +01:00
Jan Eitzinger
97f391deff GraphQL remove toplevel thresholds and add Unit
Add Unit to metric values in subCluster
2023-03-10 12:13:40 +01:00
Christoph Kluge
a651219a2c
Merge pull request #96 from ClusterCockpit/92_add_shortjobduration_config
Move ShortRunningJobsDuration value to root config
2023-03-09 16:43:43 +01:00
Christoph Kluge
14665df439 Better test array 2023-03-06 16:35:14 +01:00
Christoph Kluge
cae7257673 Add tests for role checks, update test.db 2023-03-06 16:32:58 +01:00
Christoph Kluge
f37e7c26f6 Rework roles as enum, change AuthSource to enum 2023-03-06 11:44:38 +01:00
Christoph Kluge
4f1f034702 Move ShortRunningJobsDuration value to root config 2023-03-01 16:34:59 +01:00
9d28cb83f0 Remove swagger enum comment and rebuild Swagger docs
Relates to #95
2023-03-01 10:49:08 +01:00
Christoph Kluge
df44bd9621 Merge branch 'master' into 40_45_82_update_roles 2023-02-28 11:18:46 +01:00
1fc090965d Add missing error code 2023-02-27 09:16:18 +01:00
e13f184414 Regenerate Swagger docs
Relates to #94 and #95
2023-02-27 07:49:37 +01:00
89b1d8f7da Treat error no change as non fatal 2023-02-25 08:26:56 +01:00
ae5a73335d Fix bugs in getJobs REST Api endpoint
Relates to #94 and #95
2023-02-25 07:21:54 +01:00
Christoph Kluge
8bd72ce807 Small fixes after full file review
- Remove unnecessary field 'project' from GQl and regenerate
- Add newlines to file ends
- Fix command-line manager user addition
2023-02-23 12:33:14 +01:00
Christoph Kluge
d30dba109d Fix test 2023-02-22 16:57:15 +01:00
Christoph Kluge
365888c645 Fix tag panic and simplify code 2023-02-22 16:49:23 +01:00
Christoph Kluge
d6b8d711f6 Rework searchbar functions, add user jobtable 2023-02-22 16:30:01 +01:00
Christoph Kluge
d0d578a1e6 Fix user.projects up migration file 2023-02-22 10:17:40 +01:00
Christoph Kluge
48728e64c4 Merge branch 'master' into 40_45_82_update_roles 2023-02-22 09:28:41 +01:00
60437ce264 Rebuild GraphQL api and Swagger UI code. 2023-02-21 18:13:16 +01:00
Christoph Kluge
822c8b570c Remove exists checks from sql migrations to v3 2023-02-21 18:05:29 +01:00
Christoph Kluge
3b7ca17b3c Merge branch 'master' into 40_45_82_update_roles 2023-02-21 17:36:58 +01:00
Christoph Kluge
acbef39045 Fix header, add migration for user.projects column 2023-02-21 17:33:27 +01:00
af26331f59 Merge back changes from main 2023-02-21 17:18:54 +01:00
Christoph Kluge
e0e51813ad Merge branch 'master' into 40_45_82_update_roles 2023-02-21 17:17:41 +01:00
Jan Eitzinger
33b20620ee
Merge branch 'master' into sql-repository-opt 2023-02-21 16:21:47 +01:00
999609a17a Add backend type and adopt cast expressions
Fixes #67
2023-02-21 15:22:05 +01:00
63f8555af7 Add migration support for mysql 2023-02-21 14:45:59 +01:00
d429327ce5 Fix unit test 2023-02-21 13:54:42 +01:00
42d3318125 Add documentation in README 2023-02-21 13:50:15 +01:00
5ff88e1701 Add routine names to sql query timer 2023-02-21 12:08:06 +01:00
a4f5b139f9 Remove hardcoded index generation 2023-02-21 11:36:23 +01:00
8ffb562d6b Introduce db migration support 2023-02-21 10:57:22 +01:00
643bd3fb21 Add timer calls to repository functions 2023-02-20 15:08:23 +01:00
Christoph Kluge
68efe871c7 Centralize project filter for manager role
- Remove all unnecessary frontend code for managerfilters
2023-02-20 11:24:18 +01:00
Christoph Kluge
09bcf9f355 Better implementation for querying multiple users 2023-02-20 10:20:08 +01:00
Christoph Kluge
397ab08b3b Add support for multiple projects per manager
- Handled like roles in admin view
- !! NEW COLUMN CHANGED TO "projects"
2023-02-17 15:45:31 +01:00
Christoph Kluge
bf6ec1bc98 Add search for user names, add names in user table
- Regenerated GraphQL due to new required query 'multiUser'
2023-02-17 10:45:27 +01:00
3b38d8042e Fix tests 2023-02-15 13:07:19 +01:00
033598a656 Remove loglevel notice 2023-02-15 11:50:51 +01:00
491baafd1d Add index. Formatting 2023-02-15 09:50:27 +01:00
51b2a7f30e Relocate JobStatistics from GraphQL resolver to JobRepository 2023-02-13 13:53:24 +01:00
aef821fa7e Add SQL hook for logging and timing 2023-02-13 13:52:17 +01:00
Christoph Kluge
728531946a
Merge branch 'master' into 26_improve_logmessages 2023-02-09 13:52:58 +01:00