feba722a53
Refactor and Cleanup
...
Add SecurityCheck
2023-06-22 06:26:19 +02:00
bb7c1005c9
Refactor query logic and move to job.go
2023-06-20 15:52:16 +02:00
Pay Giesselmann
a9544f5609
lower log level for frequent messages
2023-06-20 15:47:38 +02:00
Jan Eitzinger
d0516f12b0
Merge pull request #154 from ClusterCockpit/hotfix
...
Hotfix
2023-06-20 13:00:00 +02:00
c973a29734
Restore api role access in SecurityCheck
2023-06-20 12:54:26 +02:00
33613cdda0
Sync commit
2023-06-20 10:38:53 +02:00
17113eddca
Fix failed test due to config change
2023-06-15 11:23:50 +02:00
9137931b74
Add cpu_load and mem_used to Job stats
...
Fixes #111
2023-06-15 11:07:48 +02:00
Christoph Kluge
f31d8a3af0
fix test
2023-06-12 11:58:10 +02:00
Christoph Kluge
0d2e20e9e4
Handle users with no roles as "user role"
...
-for backwards compatibility
2023-06-12 11:35:16 +02:00
911dcb6626
Improve Compression Service
...
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
918f1993f2
Add loglevel access. Fix timer labels.
2023-06-09 15:02:22 +02:00
Christoph Kluge
edb1b47281
Add AddJobCount() for analysis view
2023-06-09 13:15:25 +02:00
616095fe66
Add additional job stats, fix test
2023-06-09 11:29:07 +02:00
5ba6f0ed3a
Refactor and adapt to new API
2023-06-09 09:09:41 +02:00
02752f52e4
Merge branch 'master' into refactor-job-repository
2023-06-08 06:22:51 +02:00
fb86ebdbcc
Renaming
2023-06-08 06:18:19 +02:00
Jan Eitzinger
e244a7c85a
Merge pull request #122 from pc2/master
...
Fix database migration order
2023-06-07 17:26:58 +02:00
Jan Eitzinger
1b1e46bf01
Merge pull request #123 from ClusterCockpit/121_fix_filter_reactivity_crash
...
121 fix filter reactivity crash
2023-06-07 17:24:46 +02:00
Michael Schwarz
a183c23e25
Fix truncate call on mysql and provide step 4 migrations for mysql
2023-06-07 16:49:08 +02:00
Christoph Kluge
4e568e60cd
Del JobName from Job type, jobStats name required
...
- Regenerated GQL
- Relates to #121
- return jobStats name as string
2023-06-07 14:13:59 +02:00
becb2bfa3a
Refactor Jobs stats resolver
2023-06-07 11:58:58 +02:00
Michael Schwarz
df240c2394
Fix database migration order
2023-06-07 11:45:58 +02:00
695e8e270a
Split job Repository in multiple files
2023-06-06 10:27:55 +02:00
Jan Eitzinger
797d5da80b
Merge pull request #120 from ClusterCockpit/optimize-sqlite-db-setup
...
Enable SQL hooks for statement debugging
2023-06-05 16:41:52 +02:00
96d036a128
Enable SQL hooks for statement debugging
2023-06-05 16:38:58 +02:00
Jan Eitzinger
e32872181f
Merge pull request #118 from ClusterCockpit/optimize-sqlite-db-setup
...
Add migration for job table contraints
2023-06-05 16:13:24 +02:00
d53ef526dd
Fix tests
2023-06-05 16:10:47 +02:00
e444b83149
Fix errors
2023-06-05 16:07:01 +02:00
5ca4bdbd91
Remove VACUUM
2023-06-05 16:01:14 +02:00
8b6a1c8045
Add migration for job table contraints
2023-06-05 15:58:07 +02:00
Christoph Kluge
a6cb833843
Merge branch 'master' into 97_107_mark_and_show_shared
...
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
2de85b0120
Add composite indexes
2023-06-01 15:40:37 +02:00
203ccdf73b
Rename test db
2023-06-01 15:29:25 +02:00
447fc574e8
Add query benchmark
2023-06-01 15:27:19 +02:00
Christoph Kluge
37d5db443f
Merge branch 'master' into 97_107_mark_and_show_shared
2023-06-01 15:24:26 +02:00
205a400c6f
Also Benchmarks need _test in name
2023-06-01 14:10:10 +02:00
012b131b23
Refactor for benchmarking
2023-06-01 13:57:35 +02:00
0f87f651f9
Refactor DB connection settings
2023-06-01 08:03:12 +02:00
Christoph Kluge
270750a78d
Move concurrentJobs as field to main job query
2023-05-16 12:42:06 +02:00
19e3ba7290
Improve retention
...
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
538427d59b
Introduce Retention and compression
2023-05-09 16:33:26 +02:00
788d2cd61c
Fix repo tests
2023-05-05 06:02:17 +02:00
7491ae9e3c
Fix typo in DB schema
2023-05-04 17:26:33 +02:00
066c4553b4
Refactor
2023-05-04 16:27:30 +02:00
5404be3ca7
Add unique constrain on job table
2023-05-04 16:04:57 +02:00
c5b67d64d2
Add Table flush on init
2023-05-04 16:03:04 +02:00
d9b18d1763
Cleanup
...
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
27894ea0a9
Restructure tests
2023-05-04 07:00:30 +02:00
Christoph Kluge
d93610f700
List parallel jobs on node for jobs on shared node
...
- Relates to issue #97
- required GQL schema extension and regeneration
- Works for archived jobs aswell
2023-04-28 12:34:40 +02:00
ebba4371eb
Extract importer package and restructure tests
2023-04-28 08:49:58 +02:00
5a09277d1c
Restructure black box test
...
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25
Synchronize DB Schema with json schema
...
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
10f4e09562
Set all versions and print in version flag
2023-04-12 10:43:46 +02:00
Jan Eitzinger
f8ba79e9e7
Merge branch 'master' into import-data-sanitation
2023-04-07 08:57:42 +02:00
Jan Eitzinger
d420b8b666
Merge branch 'master' into 40_45_82_update_roles
2023-04-07 08:19:04 +02:00
Jan Eitzinger
559445900d
Adapt job iterator
...
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
5d535edb48
Synchronize go job struct with schema
2023-03-29 06:46:33 +02:00
Christoph Kluge
de71fee61e
Merge pull request #98 from giesselmann/fix-updateconfig
...
Fix UpdateConfig
2023-03-21 12:09:40 +01:00
Pay Giesselmann
de822e13fd
Fix UpdateConfig
2023-03-13 10:45:55 +01:00
Christoph Kluge
f37e7c26f6
Rework roles as enum, change AuthSource to enum
2023-03-06 11:44:38 +01:00
Christoph Kluge
4f1f034702
Move ShortRunningJobsDuration value to root config
2023-03-01 16:34:59 +01:00
Christoph Kluge
df44bd9621
Merge branch 'master' into 40_45_82_update_roles
2023-02-28 11:18:46 +01:00
89b1d8f7da
Treat error no change as non fatal
2023-02-25 08:26:56 +01:00
Christoph Kluge
8bd72ce807
Small fixes after full file review
...
- Remove unnecessary field 'project' from GQl and regenerate
- Add newlines to file ends
- Fix command-line manager user addition
2023-02-23 12:33:14 +01:00
Christoph Kluge
d30dba109d
Fix test
2023-02-22 16:57:15 +01:00
Christoph Kluge
365888c645
Fix tag panic and simplify code
2023-02-22 16:49:23 +01:00
Christoph Kluge
d6b8d711f6
Rework searchbar functions, add user jobtable
2023-02-22 16:30:01 +01:00
Christoph Kluge
d0d578a1e6
Fix user.projects up migration file
2023-02-22 10:17:40 +01:00
Christoph Kluge
48728e64c4
Merge branch 'master' into 40_45_82_update_roles
2023-02-22 09:28:41 +01:00
60437ce264
Rebuild GraphQL api and Swagger UI code.
2023-02-21 18:13:16 +01:00
Christoph Kluge
822c8b570c
Remove exists checks from sql migrations to v3
2023-02-21 18:05:29 +01:00
Christoph Kluge
3b7ca17b3c
Merge branch 'master' into 40_45_82_update_roles
2023-02-21 17:36:58 +01:00
Christoph Kluge
acbef39045
Fix header, add migration for user.projects column
2023-02-21 17:33:27 +01:00
af26331f59
Merge back changes from main
2023-02-21 17:18:54 +01:00
Christoph Kluge
e0e51813ad
Merge branch 'master' into 40_45_82_update_roles
2023-02-21 17:17:41 +01:00
Jan Eitzinger
33b20620ee
Merge branch 'master' into sql-repository-opt
2023-02-21 16:21:47 +01:00
999609a17a
Add backend type and adopt cast expressions
...
Fixes #67
2023-02-21 15:22:05 +01:00
63f8555af7
Add migration support for mysql
2023-02-21 14:45:59 +01:00
42d3318125
Add documentation in README
2023-02-21 13:50:15 +01:00
5ff88e1701
Add routine names to sql query timer
2023-02-21 12:08:06 +01:00
a4f5b139f9
Remove hardcoded index generation
2023-02-21 11:36:23 +01:00
8ffb562d6b
Introduce db migration support
2023-02-21 10:57:22 +01:00
643bd3fb21
Add timer calls to repository functions
2023-02-20 15:08:23 +01:00
Christoph Kluge
68efe871c7
Centralize project filter for manager role
...
- Remove all unnecessary frontend code for managerfilters
2023-02-20 11:24:18 +01:00
Christoph Kluge
09bcf9f355
Better implementation for querying multiple users
2023-02-20 10:20:08 +01:00
Christoph Kluge
397ab08b3b
Add support for multiple projects per manager
...
- Handled like roles in admin view
- !! NEW COLUMN CHANGED TO "projects"
2023-02-17 15:45:31 +01:00
Christoph Kluge
bf6ec1bc98
Add search for user names, add names in user table
...
- Regenerated GraphQL due to new required query 'multiUser'
2023-02-17 10:45:27 +01:00
3b38d8042e
Fix tests
2023-02-15 13:07:19 +01:00
033598a656
Remove loglevel notice
2023-02-15 11:50:51 +01:00
491baafd1d
Add index. Formatting
2023-02-15 09:50:27 +01:00
51b2a7f30e
Relocate JobStatistics from GraphQL resolver to JobRepository
2023-02-13 13:53:24 +01:00
aef821fa7e
Add SQL hook for logging and timing
2023-02-13 13:52:17 +01:00
Christoph Kluge
728531946a
Merge branch 'master' into 26_improve_logmessages
2023-02-09 13:52:58 +01:00
Jan Eitzinger
180e9cb09d
Merge pull request #80 from ClusterCockpit/34-52-rework-searchbar
...
Merge Searchbar Rework
2023-02-09 11:38:02 +01:00
Christoph Kluge
ef1b2d7a81
Fix typo
2023-02-01 12:29:58 +01:00
Christoph Kluge
a885e69125
Adapt loglevel for logs, shorten strings, fix formats, streamline
...
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5
Add log messages to error events w/o log message, primaryly error level
...
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
7d4f4ab2c8
Merge branch 'master' into 40_45_82_update_roles
2023-01-30 14:33:44 +01:00
Christoph Kluge
186c8f7311
Fix user display configrations not persisting in SQLite table
...
- Was missing user.username key
2023-01-30 14:33:12 +01:00
Christoph Kluge
dd5e0682a1
Fix test file, add suer table project column to test.db
2023-01-30 14:03:44 +01:00
Christoph Kluge
c4d91039e2
Add support role to tag count
...
- works equal to admin
2023-01-30 13:48:27 +01:00
Christoph Kluge
2590ce7e44
Adapt count tags for managers
2023-01-30 13:43:12 +01:00
Christoph Kluge
b2aed2f16b
Add 'project' to user table, add 'manager' role, conditional web render
...
- Addresses issues #40 #45 #82
- Reworked Navigation Header for all roles
- 'Manager' role added, can be assigned a project-id in config by admins
- BREAKING! -> Added 'project' column in SQLite3 table 'user'
- Manager-Assigned project will be added to all graphql filters: Only show Jobs and Users of given project
- 'My Jobs' Tab for all Roles
- Switched from Bool "isAdmin" to integer authLevels
- Removed critical data frontend logging
- Reworked repo.query.SecurityCheck()
2023-01-27 18:36:58 +01:00
Christoph Kluge
834f9d9085
Add role helper functions, add project role barebone, add valid role arr
...
- HasAnyRoles([]string): Checks if user has *one* of the roles
- HasAllRoles([]string): Cheks if user has *all* of the roles
- HasNotRoles([]string): Checks if user has *none* of the roles
- IsValidRole(string): Checks if given string is known valid role
2023-01-25 16:59:16 +01:00
Christoph Kluge
bd770d0e32
Fix return error, fix wrong log path, add notice log where applicable
...
- Adresses issue #26
2023-01-24 12:02:29 +01:00
Christoph Kluge
79a949b55e
Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
...
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19
add more information to existing errors logs and panics
...
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Christoph Kluge
1873c6a87d
Merge branch 'master' into 34-52-rework-searchbar
2023-01-13 09:58:56 +01:00
Christoph Kluge
7998577076
Remove metadata from scanJob
...
- Leftover from initial jobName query implementation tryouts: unnecessary
2023-01-12 11:57:44 +01:00
Christoph Kluge
84a1c4bdaf
fix wrong vartype for log
2023-01-12 11:34:21 +01:00
Christoph Kluge
bad3f9ea95
Makeover of searchbar, add "not found" results-redirects
...
- Tagged Search has priority
- Best guess search if no tag provided
- "Hard" errors only on searchtag-malformed or searchtag-unknown, or if best guess search fails
2023-01-12 11:26:01 +01:00
Christoph Kluge
a642e9cc7c
Add working, but manually entered jobName url query
2023-01-11 16:25:02 +01:00
Christoph Kluge
c9ee8b552d
Rework search to show jobId matches in table, add projectId search
2022-12-14 10:02:22 +01:00
Christoph Kluge
ac8548a950
Initial orientation
2022-12-13 09:53:37 +01:00
Pay Giesselmann
c4bdff2210
do not close channel on wait
2022-12-08 16:21:56 +01:00
Pay Giesselmann
a0f7a598ad
archive jobs in single worker thread
2022-12-08 15:04:58 +01:00
Jan Eitzinger
fd16a1b637
Fix cnt query scan. Cosmetic changes.
2022-11-25 15:15:05 +01:00
Jan Eitzinger
948da8f10b
Review Sagger config. Delete Job endpoints
2022-11-11 15:26:27 +01:00
Joachim Meyer
574d941c6c
Upon start job, compare start time against all matching jobs.
...
For requeued jobs, we might have multiple jobs with matching cluster and job id, but differing start time. So check all of the matching ones against the start time.
2022-11-09 11:49:36 +01:00
Christoph Kluge
ec772f5aff
Fix the fix
...
Note: Changes should be removed before merge - Fix to allow compilation only
2022-10-05 14:09:11 +02:00
Christoph Kluge
bb0f8a2905
Fix compilation
...
- Unit.Prefix prop used in assignement instead if Unit type
2022-10-05 11:16:43 +02:00
Jan Eitzinger
29d215fcea
Intermediate Save commit
2022-10-04 10:12:35 +02:00
Jan Eitzinger
7251344d4a
Add convenience routines to unit package
2022-09-28 16:25:26 +02:00
Jan Eitzinger
918a07735d
Merge branch 'master' into dev-job-archive-module
2022-09-26 15:12:46 +02:00
Jan Eitzinger
9ae7a813fa
Restructure import job test due to db handle clash
2022-09-23 16:11:35 +02:00
Jan Eitzinger
5c715b6ec2
Add test for import job handler. Improve error messages.
2022-09-23 15:23:45 +02:00
Jan Eitzinger
35114aeb2d
Cleanup. Fix failing test
2022-09-20 13:07:46 +02:00
Jan Eitzinger
520c814e3b
Introduce import job flag
2022-09-13 15:20:07 +02:00
Jan Eitzinger
00fc5f4774
Remove to set ArrayJobId on DB Init
2022-09-12 14:51:51 +02:00
Jan Eitzinger
cca0722b01
Remove obsolete variable
2022-09-12 14:46:18 +02:00
Jan Eitzinger
10c6b366a5
Refactor UserCfg, add test
2022-09-12 13:34:21 +02:00
Jan Eitzinger
c64a935750
Cleanup and improve error handling
2022-09-11 07:13:08 +02:00
Jan Eitzinger
8856f26fb0
Reformat and Refactor packages. Rebuild GraphQL.
2022-09-07 12:24:45 +02:00
Jan Eitzinger
fc76eed899
Refactor package structure
...
Builds but not tested
2022-09-05 17:46:38 +02:00
Michael Schwarz
84bac7e520
Add new role support. This enables designated users to see all jobs.
2022-08-25 12:45:11 +02:00
Jan Eitzinger
26df1e7c14
Add copyright and license header. Update license year
2022-07-29 06:29:21 +02:00
Jan Eitzinger
8446f6267a
Use internal lrucache
2022-06-22 06:11:00 +02:00
Jan Eitzinger
81819db436
Refactor directory structure
2022-06-21 17:52:36 +02:00
