Commit Graph

216 Commits

Author SHA1 Message Date
695e8e270a Split job Repository in multiple files 2023-06-06 10:27:55 +02:00
Jan Eitzinger
797d5da80b
Merge pull request #120 from ClusterCockpit/optimize-sqlite-db-setup
Enable SQL hooks for statement debugging
2023-06-05 16:41:52 +02:00
96d036a128 Enable SQL hooks for statement debugging 2023-06-05 16:38:58 +02:00
Jan Eitzinger
e32872181f
Merge pull request #118 from ClusterCockpit/optimize-sqlite-db-setup
Add migration for job table contraints
2023-06-05 16:13:24 +02:00
d53ef526dd Fix tests 2023-06-05 16:10:47 +02:00
e444b83149 Fix errors 2023-06-05 16:07:01 +02:00
5ca4bdbd91 Remove VACUUM 2023-06-05 16:01:14 +02:00
8b6a1c8045 Add migration for job table contraints 2023-06-05 15:58:07 +02:00
Christoph Kluge
a6cb833843 Merge branch 'master' into 97_107_mark_and_show_shared
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
2de85b0120 Add composite indexes 2023-06-01 15:40:37 +02:00
203ccdf73b Rename test db 2023-06-01 15:29:25 +02:00
447fc574e8 Add query benchmark 2023-06-01 15:27:19 +02:00
Christoph Kluge
37d5db443f Merge branch 'master' into 97_107_mark_and_show_shared 2023-06-01 15:24:26 +02:00
205a400c6f Also Benchmarks need _test in name 2023-06-01 14:10:10 +02:00
012b131b23 Refactor for benchmarking 2023-06-01 13:57:35 +02:00
0f87f651f9 Refactor DB connection settings 2023-06-01 08:03:12 +02:00
Christoph Kluge
270750a78d Move concurrentJobs as field to main job query 2023-05-16 12:42:06 +02:00
19e3ba7290 Improve retention
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
538427d59b Introduce Retention and compression 2023-05-09 16:33:26 +02:00
788d2cd61c Fix repo tests 2023-05-05 06:02:17 +02:00
7491ae9e3c Fix typo in DB schema 2023-05-04 17:26:33 +02:00
066c4553b4 Refactor 2023-05-04 16:27:30 +02:00
5404be3ca7 Add unique constrain on job table 2023-05-04 16:04:57 +02:00
c5b67d64d2 Add Table flush on init 2023-05-04 16:03:04 +02:00
d9b18d1763 Cleanup
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
27894ea0a9 Restructure tests 2023-05-04 07:00:30 +02:00
Christoph Kluge
d93610f700 List parallel jobs on node for jobs on shared node
- Relates to issue #97
- required GQL schema extension and regeneration
- Works for archived jobs aswell
2023-04-28 12:34:40 +02:00
ebba4371eb Extract importer package and restructure tests 2023-04-28 08:49:58 +02:00
5a09277d1c Restructure black box test
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25 Synchronize DB Schema with json schema
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
10f4e09562 Set all versions and print in version flag 2023-04-12 10:43:46 +02:00
Jan Eitzinger
f8ba79e9e7 Merge branch 'master' into import-data-sanitation 2023-04-07 08:57:42 +02:00
Jan Eitzinger
d420b8b666
Merge branch 'master' into 40_45_82_update_roles 2023-04-07 08:19:04 +02:00
Jan Eitzinger
559445900d Adapt job iterator
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
5d535edb48 Synchronize go job struct with schema 2023-03-29 06:46:33 +02:00
Christoph Kluge
de71fee61e
Merge pull request #98 from giesselmann/fix-updateconfig
Fix UpdateConfig
2023-03-21 12:09:40 +01:00
Pay Giesselmann
de822e13fd
Fix UpdateConfig 2023-03-13 10:45:55 +01:00
Christoph Kluge
f37e7c26f6 Rework roles as enum, change AuthSource to enum 2023-03-06 11:44:38 +01:00
Christoph Kluge
4f1f034702 Move ShortRunningJobsDuration value to root config 2023-03-01 16:34:59 +01:00
Christoph Kluge
df44bd9621 Merge branch 'master' into 40_45_82_update_roles 2023-02-28 11:18:46 +01:00
89b1d8f7da Treat error no change as non fatal 2023-02-25 08:26:56 +01:00
Christoph Kluge
8bd72ce807 Small fixes after full file review
- Remove unnecessary field 'project' from GQl and regenerate
- Add newlines to file ends
- Fix command-line manager user addition
2023-02-23 12:33:14 +01:00
Christoph Kluge
d30dba109d Fix test 2023-02-22 16:57:15 +01:00
Christoph Kluge
365888c645 Fix tag panic and simplify code 2023-02-22 16:49:23 +01:00
Christoph Kluge
d6b8d711f6 Rework searchbar functions, add user jobtable 2023-02-22 16:30:01 +01:00
Christoph Kluge
d0d578a1e6 Fix user.projects up migration file 2023-02-22 10:17:40 +01:00
Christoph Kluge
48728e64c4 Merge branch 'master' into 40_45_82_update_roles 2023-02-22 09:28:41 +01:00
60437ce264 Rebuild GraphQL api and Swagger UI code. 2023-02-21 18:13:16 +01:00
Christoph Kluge
822c8b570c Remove exists checks from sql migrations to v3 2023-02-21 18:05:29 +01:00
Christoph Kluge
3b7ca17b3c Merge branch 'master' into 40_45_82_update_roles 2023-02-21 17:36:58 +01:00
Christoph Kluge
acbef39045 Fix header, add migration for user.projects column 2023-02-21 17:33:27 +01:00
af26331f59 Merge back changes from main 2023-02-21 17:18:54 +01:00
Christoph Kluge
e0e51813ad Merge branch 'master' into 40_45_82_update_roles 2023-02-21 17:17:41 +01:00
Jan Eitzinger
33b20620ee
Merge branch 'master' into sql-repository-opt 2023-02-21 16:21:47 +01:00
999609a17a Add backend type and adopt cast expressions
Fixes #67
2023-02-21 15:22:05 +01:00
63f8555af7 Add migration support for mysql 2023-02-21 14:45:59 +01:00
42d3318125 Add documentation in README 2023-02-21 13:50:15 +01:00
5ff88e1701 Add routine names to sql query timer 2023-02-21 12:08:06 +01:00
a4f5b139f9 Remove hardcoded index generation 2023-02-21 11:36:23 +01:00
8ffb562d6b Introduce db migration support 2023-02-21 10:57:22 +01:00
643bd3fb21 Add timer calls to repository functions 2023-02-20 15:08:23 +01:00
Christoph Kluge
68efe871c7 Centralize project filter for manager role
- Remove all unnecessary frontend code for managerfilters
2023-02-20 11:24:18 +01:00
Christoph Kluge
09bcf9f355 Better implementation for querying multiple users 2023-02-20 10:20:08 +01:00
Christoph Kluge
397ab08b3b Add support for multiple projects per manager
- Handled like roles in admin view
- !! NEW COLUMN CHANGED TO "projects"
2023-02-17 15:45:31 +01:00
Christoph Kluge
bf6ec1bc98 Add search for user names, add names in user table
- Regenerated GraphQL due to new required query 'multiUser'
2023-02-17 10:45:27 +01:00
3b38d8042e Fix tests 2023-02-15 13:07:19 +01:00
033598a656 Remove loglevel notice 2023-02-15 11:50:51 +01:00
491baafd1d Add index. Formatting 2023-02-15 09:50:27 +01:00
51b2a7f30e Relocate JobStatistics from GraphQL resolver to JobRepository 2023-02-13 13:53:24 +01:00
aef821fa7e Add SQL hook for logging and timing 2023-02-13 13:52:17 +01:00
Christoph Kluge
728531946a
Merge branch 'master' into 26_improve_logmessages 2023-02-09 13:52:58 +01:00
Jan Eitzinger
180e9cb09d
Merge pull request #80 from ClusterCockpit/34-52-rework-searchbar
Merge Searchbar Rework
2023-02-09 11:38:02 +01:00
Christoph Kluge
ef1b2d7a81 Fix typo 2023-02-01 12:29:58 +01:00
Christoph Kluge
a885e69125 Adapt loglevel for logs, shorten strings, fix formats, streamline
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5 Add log messages to error events w/o log message, primaryly error level
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
7d4f4ab2c8 Merge branch 'master' into 40_45_82_update_roles 2023-01-30 14:33:44 +01:00
Christoph Kluge
186c8f7311 Fix user display configrations not persisting in SQLite table
- Was missing user.username key
2023-01-30 14:33:12 +01:00
Christoph Kluge
dd5e0682a1 Fix test file, add suer table project column to test.db 2023-01-30 14:03:44 +01:00
Christoph Kluge
c4d91039e2 Add support role to tag count
- works equal to admin
2023-01-30 13:48:27 +01:00
Christoph Kluge
2590ce7e44 Adapt count tags for managers 2023-01-30 13:43:12 +01:00
Christoph Kluge
b2aed2f16b Add 'project' to user table, add 'manager' role, conditional web render
- Addresses issues #40 #45 #82
- Reworked Navigation Header for all roles
- 'Manager' role added, can be assigned a project-id in config by admins
- BREAKING! -> Added 'project' column in SQLite3 table 'user'
- Manager-Assigned project will be added to all graphql filters: Only show Jobs and Users of given project
- 'My Jobs' Tab for all Roles
- Switched from Bool "isAdmin" to integer authLevels
- Removed critical data frontend logging
- Reworked repo.query.SecurityCheck()
2023-01-27 18:36:58 +01:00
Christoph Kluge
834f9d9085 Add role helper functions, add project role barebone, add valid role arr
- HasAnyRoles([]string): Checks if user has *one* of the roles
- HasAllRoles([]string): Cheks if user has *all* of the roles
- HasNotRoles([]string): Checks if user has *none* of the roles
- IsValidRole(string): Checks if given string is known valid role
2023-01-25 16:59:16 +01:00
Christoph Kluge
bd770d0e32 Fix return error, fix wrong log path, add notice log where applicable
- Adresses issue #26
2023-01-24 12:02:29 +01:00
Christoph Kluge
79a949b55e Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19 add more information to existing errors logs and panics
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Christoph Kluge
1873c6a87d
Merge branch 'master' into 34-52-rework-searchbar 2023-01-13 09:58:56 +01:00
Christoph Kluge
7998577076 Remove metadata from scanJob
- Leftover from initial jobName query implementation tryouts: unnecessary
2023-01-12 11:57:44 +01:00
Christoph Kluge
84a1c4bdaf fix wrong vartype for log 2023-01-12 11:34:21 +01:00
Christoph Kluge
bad3f9ea95 Makeover of searchbar, add "not found" results-redirects
- Tagged Search has priority
- Best guess search if no tag provided
- "Hard" errors only on searchtag-malformed or searchtag-unknown, or if best guess search fails
2023-01-12 11:26:01 +01:00
Christoph Kluge
a642e9cc7c Add working, but manually entered jobName url query 2023-01-11 16:25:02 +01:00
Christoph Kluge
c9ee8b552d Rework search to show jobId matches in table, add projectId search 2022-12-14 10:02:22 +01:00
Christoph Kluge
ac8548a950 Initial orientation 2022-12-13 09:53:37 +01:00
Pay Giesselmann
c4bdff2210 do not close channel on wait 2022-12-08 16:21:56 +01:00
Pay Giesselmann
a0f7a598ad archive jobs in single worker thread 2022-12-08 15:04:58 +01:00
Jan Eitzinger
fd16a1b637 Fix cnt query scan. Cosmetic changes. 2022-11-25 15:15:05 +01:00
Jan Eitzinger
948da8f10b Review Sagger config. Delete Job endpoints 2022-11-11 15:26:27 +01:00
Joachim Meyer
574d941c6c Upon start job, compare start time against all matching jobs.
For requeued jobs, we might have multiple jobs with matching cluster and job id, but differing start time. So check all of the matching ones against the start time.
2022-11-09 11:49:36 +01:00
Christoph Kluge
ec772f5aff Fix the fix
Note: Changes should be removed before merge - Fix to allow compilation only
2022-10-05 14:09:11 +02:00
Christoph Kluge
bb0f8a2905 Fix compilation
- Unit.Prefix prop used in assignement instead if Unit type
2022-10-05 11:16:43 +02:00
Jan Eitzinger
29d215fcea Intermediate Save commit 2022-10-04 10:12:35 +02:00
Jan Eitzinger
7251344d4a Add convenience routines to unit package 2022-09-28 16:25:26 +02:00
Jan Eitzinger
918a07735d
Merge branch 'master' into dev-job-archive-module 2022-09-26 15:12:46 +02:00
Jan Eitzinger
9ae7a813fa Restructure import job test due to db handle clash 2022-09-23 16:11:35 +02:00
Jan Eitzinger
5c715b6ec2 Add test for import job handler. Improve error messages. 2022-09-23 15:23:45 +02:00
Jan Eitzinger
35114aeb2d Cleanup. Fix failing test 2022-09-20 13:07:46 +02:00
Jan Eitzinger
520c814e3b Introduce import job flag 2022-09-13 15:20:07 +02:00
Jan Eitzinger
00fc5f4774 Remove to set ArrayJobId on DB Init 2022-09-12 14:51:51 +02:00
Jan Eitzinger
cca0722b01 Remove obsolete variable 2022-09-12 14:46:18 +02:00
Jan Eitzinger
10c6b366a5 Refactor UserCfg, add test 2022-09-12 13:34:21 +02:00
Jan Eitzinger
c64a935750 Cleanup and improve error handling 2022-09-11 07:13:08 +02:00
Jan Eitzinger
8856f26fb0 Reformat and Refactor packages. Rebuild GraphQL. 2022-09-07 12:24:45 +02:00
Jan Eitzinger
fc76eed899 Refactor package structure
Builds but not tested
2022-09-05 17:46:38 +02:00
Michael Schwarz
84bac7e520 Add new role support. This enables designated users to see all jobs. 2022-08-25 12:45:11 +02:00
Jan Eitzinger
26df1e7c14 Add copyright and license header. Update license year 2022-07-29 06:29:21 +02:00
Jan Eitzinger
8446f6267a Use internal lrucache 2022-06-22 06:11:00 +02:00
Jan Eitzinger
81819db436 Refactor directory structure 2022-06-21 17:52:36 +02:00