Commit Graph

498 Commits

Author SHA1 Message Date
Christoph Kluge
68cf952ac6 Merge branch 'Refactor-job-footprint' of https://github.com/ClusterCockpit/cc-backend into Refactor-job-footprint 2024-07-11 17:33:21 +02:00
Christoph Kluge
e14d6a81fe fix: fix db migration to v8, changes key name to cpu_load 2024-07-11 17:24:33 +02:00
Christoph Kluge
a4912893a8 Frontend refactor backend changes 2024-07-11 17:23:59 +02:00
b64ce1f67f
Add LowerIsBetter Metric boolean. Upgrade dependencies. 2024-07-11 16:58:12 +02:00
f1427d5272
Add global metric list including graphQL query 2024-07-11 11:09:14 +02:00
Christoph Kluge
bf6b87d65c Fix circular import after merge 2024-07-09 09:50:32 +02:00
Christoph Kluge
0240997257 Merge branch '263_use_median_for_statsseries' into Refactor-job-footprint 2024-07-09 09:28:21 +02:00
Christoph Kluge
f1e341f0b9 Initial commit for frontend refactor 2024-07-09 09:17:50 +02:00
Christoph Kluge
11176da5d8 Merge branch 'Refactor-job-footprint' into 264_user_api_access 2024-07-05 16:11:42 +02:00
Christoph Kluge
0a604336c4 Fix other apitest subtests 2024-07-05 15:42:08 +02:00
Christoph Kluge
be9df7649f fix: setup user in api test config 2024-07-05 15:25:24 +02:00
Christoph Kluge
63fb923995 fix: fix api test router init 2024-07-05 13:16:21 +02:00
Christoph Kluge
3afe40083d rename api userconfig to frontend, return json on api auth error 2024-07-05 11:48:06 +02:00
ac9bba8b5b
Restructure and simplify job repo 2024-07-04 15:05:24 +02:00
80c46bea7f
Fix bugs and failed testcases 2024-07-04 14:14:27 +02:00
Christoph Kluge
1072d7b449 Improve auth handling of rest apis used in frontend for compatibility 2024-07-04 11:16:45 +02:00
Christoph Kluge
61eebc9fbd Rework initial commit
- moved frontend configuration api to new subrouter for compatibility
2024-07-03 17:24:26 +02:00
130613b717
Fix build errors
Code not yet functional
2024-06-28 17:08:28 +02:00
b3c1f39a0e
Merge branch 'master' into Refactor-job-footprint 2024-06-28 16:50:04 +02:00
97c807cd33
Add migration for footprint 2024-06-28 16:49:24 +02:00
786770f56a
Start to convert to new footprint layout 2024-06-28 16:48:10 +02:00
5c5484b4d2
Export package runtimeEnv 2024-06-25 07:12:46 +02:00
Christoph Kluge
70e63764ff fix: allow single partial errors on otherwise non-empty returned metric array 2024-06-13 12:38:29 +02:00
Christoph Kluge
320c87a1db fix: add additional 30d fitler to searchbar fallback handling 2024-05-27 11:11:25 +02:00
Christoph Kluge
420bec7c46 fix: fix jobname and arrayjobid timeouts by adding additional 30d filter
- improve archive worker logs
- add arrayjobid filter to url if used
2024-05-23 11:53:23 +02:00
Christoph Kluge
684cb5a376 feat: change statistics render of metric plot to min/max/median
- #263
2024-05-08 16:17:42 +02:00
Christoph Kluge
597bccc080 fix: add SQL JSON validity check to meta_data query 2024-05-06 13:15:15 +02:00
Christoph Kluge
a22340196f Fix: Improve jobName query by parsing DB field as JSON
- No DB mirgration required
- SQLite internal EXTRACT function used
2024-04-22 12:14:40 +02:00
Christoph Kluge
cbaeffde2c fix: improve speed of hasNextPage query for infinite scroll 2024-04-22 11:29:31 +02:00
b67f5436f8 Merge branch 'hotfix' of github.com:ClusterCockpit/cc-backend into hotfix 2024-04-21 15:04:09 +02:00
b637ddeb28 Refactor and reformat userConfig 2024-04-21 15:04:00 +02:00
fb8bbea99d Remove year in copyright notice 2024-04-11 23:04:30 +02:00
Jan Eitzinger
baa51db26c
Merge pull request #252 from ClusterCockpit/20_infinite_scroll
20 infinite scroll
2024-03-28 17:43:02 +01:00
6828c97415 Add central function to persist users on Login 2024-03-28 14:22:23 +01:00
50401e0030 Fix conditional rendering of OIDC button in login 2024-03-28 13:18:25 +01:00
642fd5cc91 Merge branch 'master' into 236-user-authentication-using-keycloak-or-any-openid-client-for-using-external-auth-providers-such-as-ldap-github-google 2024-03-28 12:07:58 +01:00
e8fb5a0030 Add OpenID Connect authentication
Fixes #236
Template conditional not yet working
Needs more testing
2024-03-28 12:01:13 +01:00
Christoph Kluge
0dee5073c6 fix: make hasnextpage optional parameter, use only if inf scroll configured 2024-03-26 16:27:04 +01:00
Christoph Kluge
b9b452f043 feat: prototype infinite scroll implementation 2024-03-26 15:56:07 +01:00
Christoph Kluge
1f5723a97e Merge branch 'hotfix' of https://github.com/ClusterCockpit/cc-backend into hotfix 2024-03-22 16:10:35 +01:00
Christoph Kluge
5a177c952d fix: multiple accs with identical label, cloned data for single acc
- GPU id label in job view statistic table is always the same on multi GPU jobs #239
- Multiple accelerators listed in plot despite using only one #241
2024-03-22 16:10:30 +01:00
83c38e74db Refactoring: Reduze bytesize of structs. 2024-03-22 08:59:35 +01:00
1e5f2944cf Upgrade dependencies. Port to jwt-auth v5. 2024-03-21 22:02:59 +01:00
c65694b36c Add tags and metadata to job queries. Fix query parameter handling.
Fixes #248
2024-03-19 16:18:43 +01:00
0005469101 Refactor 2024-03-19 16:16:02 +01:00
Jan Eitzinger
81fe492655
Merge pull request #247 from ClusterCockpit/hotfix
Hotfix
2024-03-15 09:30:48 +01:00
Christoph Kluge
849b7e038d Fix: make footprint display configurable app-wide
- note: requires full ui-defaults object in config
2024-03-14 15:14:19 +01:00
Michael Schwarz
ce792426e6 Disable foreign key check while updating database 2024-03-14 09:15:15 +01:00
e92e727279 Extend oidc auth provider 2024-03-13 17:09:36 +01:00
f761900a3e Add initial code for oidc authentication support 2024-03-13 09:37:12 +01:00
99d55f05f8 feat: Add cluster config endpoint to rest api 2024-03-08 16:35:30 +01:00
9fd839fad8 Add rest endpoint to get all job data
Fixes #203
2024-03-08 15:31:34 +01:00
06d01962a6 feat: Allow to revert db to previous version 2024-03-08 11:28:26 +01:00
2c2c1accb5 Allow up and down migration of database 2024-03-08 10:58:45 +01:00
105b7eabf0 Add migration and introduce dirty flag handling 2024-03-08 10:47:32 +01:00
Jan Eitzinger
de1d83e1a9
Merge branch 'master' into 231-sql-statement-syntax 2024-03-08 09:05:13 +01:00
e81e56ea1d Add rest endpoint edit_meta including helper routines
Fixes #219
2024-03-08 08:51:05 +01:00
aa6336ea1e Refactor
Reformat.
Convert to query builder.
Add descriptive error log messages.
2024-03-06 14:50:08 +01:00
dd887cbb1f Do all tag queries with query builder
Fix bug in mysql database initialization.
Fixes #231
2024-03-06 13:47:52 +01:00
5a4671b7b1 Always return on metricstore error. 2024-02-23 10:08:41 +01:00
Christoph Kluge
21dde870c6
Merge branch 'master' into 214_user_status_histograms 2024-02-08 12:26:07 +01:00
Christoph Kluge
07073e290a feat: add selectable histograms to status view 2023-12-12 16:46:03 +01:00
Christoph Kluge
119637cb9b Fix using crossjoin arguments not used 2023-12-12 15:07:23 +01:00
Christoph Kluge
ee4097a2dd Add missing filters to crossjoinquery 2023-12-11 13:55:56 +01:00
Christoph Kluge
1185737eaa Add metrics to histoselect, add userfilters
- edit struct to make only count return required
2023-12-08 12:03:04 +01:00
Christoph Kluge
ead5c54bcb Prototype completed 2023-12-05 15:30:40 +01:00
Christoph Kluge
b5b355c16c Finished backend sql query and gql resolve 2023-12-05 11:59:01 +01:00
Christoph Kluge
9bc36152d9 intermediate save
- DOES NOT COMPILE
2023-12-01 13:22:01 +01:00
Christoph Kluge
e34623b1ce Add db average stats to gql, use in footprint 2023-11-24 15:11:38 +01:00
bb1c8cc25d fix: Move name extract from token in else branch 2023-09-08 12:11:49 +02:00
4b06fa788d fix: Fix buggy logic and simplify code if ValidateUser enabled 2023-09-08 11:50:28 +02:00
7a5ccff6da fix: Remove port before IP check 2023-09-07 16:36:47 +02:00
ed5ecbd914 fix: Restructure swagger docs 2023-09-07 15:14:09 +02:00
2d4759114e Add Release Notes link to release page 2023-09-07 14:33:22 +02:00
c68b9fec42 fix: Add documentation for apiAllowedIPs option 2023-09-07 14:03:41 +02:00
Christoph Kluge
69ee19bed0 fix: include running jobs case in statsQueries 2023-09-01 10:23:14 +02:00
Christoph Kluge
cffdd055c9 change: use continue for ccms.loadStats errors 2023-08-31 15:17:40 +02:00
Christoph Kluge
64796519c6 change: use continue for rooflineHeatmap errors
- hard errors blocked rendering in frontend
2023-08-31 15:10:57 +02:00
Jan Eitzinger
b836eee1e7
Merge pull request #205 from ClusterCockpit/166_add_scopes_analysis
166 add scopes analysis
2023-08-31 12:03:37 +02:00
Christoph Kluge
b623092721 feat: persist analysis and status pie selections 2023-08-30 15:15:53 +02:00
9533f06eaf Refactor repository tests
Add context to tests.
Remove special test routines
2023-08-30 15:04:50 +02:00
Christoph Kluge
59c749a164 feat: add select to analysis view pie chart
- 'Walltime' as generic default value for top list
-  Change from nodes distribution to cores distribution
2023-08-29 17:38:17 +02:00
Christoph Kluge
3014f59cc2 feat: add new distribution plots to status view
- numCores and numAccs
2023-08-29 14:02:23 +02:00
Christoph Kluge
f933cad87f feat: add select to status view pie charts
- 'Jobs' as generic default value for top lists
- Prepare histograms for cores and accs in schema
2023-08-29 14:01:01 +02:00
Pay Giesselmann
fd94d30a8e make ldap username attribute configurable 2023-08-29 09:30:57 +02:00
Pay Giesselmann
81d9015d59 :fix delete user logging 2023-08-29 09:29:56 +02:00
Christoph Kluge
69519ec040 Add requireField cases to resolver 2023-08-28 10:19:26 +02:00
Christoph Kluge
c84b819212 Fix frontend errors
- todo: debug backend handling
2023-08-28 10:00:20 +02:00
Christoph Kluge
ce758610b6 change: implement topX query in jobsStatistics 2023-08-25 17:38:25 +02:00
d7117f3d49 Add sorting and paging to JobStatsGrouped 2023-08-25 13:14:34 +02:00
13d99a6ae0 Fix typo in Jobstats resolver 2023-08-24 14:55:49 +02:00
3b8bcf7b32 Remove obsolete jobsCount resolver 2023-08-24 14:51:26 +02:00
a7dd3fbc0b fix bug in stats AddJobCount 2023-08-24 14:26:23 +02:00
Christoph Kluge
28609a3372 adapt core timeweight to sqlite name logic 2023-08-24 12:56:35 +02:00
Christoph Kluge
4eceab4dc7 fix: change analysis top users to core hours 2023-08-24 12:51:55 +02:00
Christoph Kluge
6a1e35107f fix: analysis metric histogram normalized by scope
- native acc metrics normalized by accHours
- native core metrics normalized by coreHours
2023-08-24 11:52:36 +02:00
Jan Eitzinger
b7aacd1b33
Merge pull request #194 from ClusterCockpit/105_modify_user_via_api
first iteraton of implementing ip-secured enpoint
2023-08-23 08:57:40 +02:00
Christoph Kluge
2f35482aff Merge branch 'master' into 166_add_scopes_analysis 2023-08-22 15:26:20 +02:00
Christoph Kluge
f36f62fb47 Improve user endpoint swagger docs 2023-08-21 12:12:28 +02:00
dc67a1f103 Update frontend and backend dependencies 2023-08-21 08:49:42 +02:00
Christoph Kluge
f6c4c963ec feat: Add users rest endpoint swagger docs 2023-08-18 17:18:31 +02:00
ebcae32e23 Update docs and cleanup 2023-08-18 15:56:11 +02:00
cda46141cc Cleanup and add wildcard for IP Filter 2023-08-18 13:03:11 +02:00
3028f60807 Reformat and add debug output 2023-08-18 11:59:16 +02:00
Christoph Kluge
734e818b19 Merge branch '189-refactor-authentication-module' of https://github.com/ClusterCockpit/cc-backend into 189-refactor-authentication-module 2023-08-18 11:17:33 +02:00
Christoph Kluge
57bda63506 Cleanup some error strings 2023-08-18 11:17:31 +02:00
da551a0bb4 Repair broken error handlng 2023-08-18 11:00:13 +02:00
32b0c8bdd7 Refactor and cleanup Auth configuration 2023-08-18 10:43:06 +02:00
14c487c9e4 Update test inputs 2023-08-18 09:31:57 +02:00
56d559fdd7 Fix bug with jwt max-age option 2023-08-18 09:19:30 +02:00
cfcf939339 Add config to jwt again 2023-08-18 08:57:56 +02:00
d51be5c308 Formatting and minor fixes 2023-08-18 08:49:25 +02:00
29552fadc3 Cleanup SyncOnLogin Handling 2023-08-17 14:02:04 +02:00
15231bc683 Cleanup and adapt to new structure 2023-08-17 12:34:30 +02:00
87ce4f63d4 Refactor auth module
Separate parts
Add user repository
Add user schema
2023-08-17 10:29:00 +02:00
80aed87415 Retry fetching user after CanLogin 2023-08-16 17:21:12 +02:00
65cf86586a Merge branch '105_modify_user_via_api' into 189-refactor-authentication-module 2023-08-16 09:46:41 +02:00
4f6d1fec68 Fix errors in ldap auth 2023-08-16 09:19:41 +02:00
202521cbfd Restructure routing and security check 2023-08-14 18:38:30 +02:00
90bdfcfbb6 Add secured subrouter for REST API
Rename IP filter option
Add array helper in util
2023-08-14 14:33:05 +02:00
42e05fc999 Merge branch '189-refactor-authentication-module' into 105_modify_user_via_api 2023-08-14 14:00:27 +02:00
Christoph Kluge
fe6de5bc68 Merge branch '189-refactor-authentication-module' of https://github.com/ClusterCockpit/cc-backend into 189-refactor-authentication-module 2023-08-14 13:52:29 +02:00
Christoph Kluge
e550e57ac0 Fix Java/Grails issued token parsing
- Tested locally until successfull login
- Initialize empty projects array
2023-08-14 13:52:26 +02:00
4a2afc7a5a Add LDAPSyncOnLogin option
Cleanup
Extend docs
Remove obsolete Expiration attribute
2023-08-14 12:40:21 +02:00
19d645f65c Readd URL token and cleanup
Fix session values.
2023-08-12 09:02:41 +02:00
b8273a9b02 refactor auth module
Restructure module
Separate JWT auth variants
Cleanup code
Fixes #189
2023-08-11 10:00:23 +02:00
Pay Giesselmann
d929bdc9a1 omit metrics with empty series 2023-08-08 10:19:37 +02:00
Christoph Kluge
6393035e55 first iteraton of implementing ip-secured enpoint 2023-08-03 17:47:09 +02:00
Christoph Kluge
f7571211fd initial branch commit 2023-07-21 16:33:53 +02:00
Jan Eitzinger
c0ab5de2f1
Merge pull request #182 from ClusterCockpit/179_fix_frontend_apiusers
Fix frontend render for users with api role
2023-07-20 07:42:15 +02:00
Joachim Meyer
650bcae6be Add arrayJobId searchbar option. 2023-07-19 09:46:48 +02:00
Jan Eitzinger
536a51b93f
Merge pull request #186 from ClusterCockpit/185-add-notification-banner
185 add notification banner
2023-07-19 09:13:15 +02:00
04e8279ae4 Change log level for JWT Cross login warning to debug 2023-07-19 09:04:27 +02:00
c6a0d442cc feat: Add optional notification banner on homepage
Fixes #185
2023-07-19 08:25:14 +02:00
Christoph Kluge
55943cacbf Fix frontend render for users with api role 2023-07-17 12:19:49 +02:00
df9fd77d06 Refactor auth and add docs
Cleanup and reformat
2023-07-05 09:50:44 +02:00
Christoph Kluge
2f471dc192 Change listQuery to jobId array, adapt filter pipe 2023-06-30 16:55:34 +02:00
Christoph Kluge
c04aea89c9 Implement node filter in frontend, fix backend
- Add running job count and link to list to single node view
2023-06-30 12:01:27 +02:00
b5a7249ad5 Fix sqlite migration for configuration 2023-06-30 09:15:58 +02:00
dd80d5af9e Change config value datatype
Fixes #167
2023-06-29 14:55:38 +02:00
7174f27a89 Add node filter and concurrent job list query 2023-06-28 13:35:41 +02:00
3828c138b8 Add init flag 2023-06-28 12:41:27 +02:00
6d8a3aa256 Fix bug in fsBackend and add tests for file utils 2023-06-28 07:39:39 +02:00
34050c8ce0 Fix argument dashes 2023-06-27 14:30:09 +02:00
eb626db9c2 Debug compression service 2023-06-27 14:29:56 +02:00
d1e3e06b10 Add insert timestamp to tag, jobtag
Fixes #43
2023-06-27 09:41:06 +02:00
8aaa5722e9 Fix errors in query logic 2023-06-26 16:48:13 +02:00
99a2888431 Fix typo in Query 2023-06-26 14:45:49 +02:00
Jan Eitzinger
7d85c0e9ad
Merge pull request #162 from ClusterCockpit/refactor-concurrent-job-resolver
Refactor concurrent job resolver
2023-06-26 14:28:00 +02:00
Christoph Kluge
f992ffc2da Fix SSL timing issues with best guess search 2023-06-23 16:09:33 +02:00