Christoph Kluge
ee4097a2dd
Add missing filters to crossjoinquery
2023-12-11 13:55:56 +01:00
Christoph Kluge
1185737eaa
Add metrics to histoselect, add userfilters
...
- edit struct to make only count return required
2023-12-08 12:03:04 +01:00
Christoph Kluge
ead5c54bcb
Prototype completed
2023-12-05 15:30:40 +01:00
Christoph Kluge
b5b355c16c
Finished backend sql query and gql resolve
2023-12-05 11:59:01 +01:00
Christoph Kluge
9bc36152d9
intermediate save
...
- DOES NOT COMPILE
2023-12-01 13:22:01 +01:00
Christoph Kluge
e34623b1ce
Add db average stats to gql, use in footprint
2023-11-24 15:11:38 +01:00
Christoph Kluge
69ee19bed0
fix: include running jobs case in statsQueries
2023-09-01 10:23:14 +02:00
Jan Eitzinger
b836eee1e7
Merge pull request #205 from ClusterCockpit/166_add_scopes_analysis
...
166 add scopes analysis
2023-08-31 12:03:37 +02:00
9533f06eaf
Refactor repository tests
...
Add context to tests.
Remove special test routines
2023-08-30 15:04:50 +02:00
Christoph Kluge
59c749a164
feat: add select to analysis view pie chart
...
- 'Walltime' as generic default value for top list
- Change from nodes distribution to cores distribution
2023-08-29 17:38:17 +02:00
Christoph Kluge
3014f59cc2
feat: add new distribution plots to status view
...
- numCores and numAccs
2023-08-29 14:02:23 +02:00
Christoph Kluge
f933cad87f
feat: add select to status view pie charts
...
- 'Jobs' as generic default value for top lists
- Prepare histograms for cores and accs in schema
2023-08-29 14:01:01 +02:00
Pay Giesselmann
81d9015d59
:fix delete user logging
2023-08-29 09:29:56 +02:00
Christoph Kluge
c84b819212
Fix frontend errors
...
- todo: debug backend handling
2023-08-28 10:00:20 +02:00
Christoph Kluge
ce758610b6
change: implement topX query in jobsStatistics
2023-08-25 17:38:25 +02:00
d7117f3d49
Add sorting and paging to JobStatsGrouped
2023-08-25 13:14:34 +02:00
3b8bcf7b32
Remove obsolete jobsCount resolver
2023-08-24 14:51:26 +02:00
a7dd3fbc0b
fix bug in stats AddJobCount
2023-08-24 14:26:23 +02:00
Christoph Kluge
4eceab4dc7
fix: change analysis top users to core hours
2023-08-24 12:51:55 +02:00
14c487c9e4
Update test inputs
2023-08-18 09:31:57 +02:00
15231bc683
Cleanup and adapt to new structure
2023-08-17 12:34:30 +02:00
87ce4f63d4
Refactor auth module
...
Separate parts
Add user repository
Add user schema
2023-08-17 10:29:00 +02:00
b8273a9b02
refactor auth module
...
Restructure module
Separate JWT auth variants
Cleanup code
Fixes #189
2023-08-11 10:00:23 +02:00
Christoph Kluge
2f471dc192
Change listQuery to jobId array, adapt filter pipe
2023-06-30 16:55:34 +02:00
b5a7249ad5
Fix sqlite migration for configuration
2023-06-30 09:15:58 +02:00
dd80d5af9e
Change config value datatype
...
Fixes #167
2023-06-29 14:55:38 +02:00
7174f27a89
Add node filter and concurrent job list query
2023-06-28 13:35:41 +02:00
3828c138b8
Add init flag
2023-06-28 12:41:27 +02:00
34050c8ce0
Fix argument dashes
2023-06-27 14:30:09 +02:00
eb626db9c2
Debug compression service
2023-06-27 14:29:56 +02:00
d1e3e06b10
Add insert timestamp to tag, jobtag
...
Fixes #43
2023-06-27 09:41:06 +02:00
8aaa5722e9
Fix errors in query logic
2023-06-26 16:48:13 +02:00
99a2888431
Fix typo in Query
2023-06-26 14:45:49 +02:00
Jan Eitzinger
7d85c0e9ad
Merge pull request #162 from ClusterCockpit/refactor-concurrent-job-resolver
...
Refactor concurrent job resolver
2023-06-26 14:28:00 +02:00
Christoph Kluge
f992ffc2da
Fix SSL timing issues with best guess search
2023-06-23 16:09:33 +02:00
Jan Eitzinger
2f6c16a7d8
Merge pull request #156 from giesselmann/log_levels
...
lower log level for frequent messages
2023-06-23 10:33:44 +02:00
Jan Eitzinger
76d3018b5c
Merge pull request #159 from ClusterCockpit/158_fix_searchbar
...
158 fix searchbar
2023-06-23 10:31:19 +02:00
Christoph Kluge
bcadb1adda
Remove errorcase from single searchterm logic
2023-06-22 10:58:36 +02:00
feba722a53
Refactor and Cleanup
...
Add SecurityCheck
2023-06-22 06:26:19 +02:00
bb7c1005c9
Refactor query logic and move to job.go
2023-06-20 15:52:16 +02:00
Pay Giesselmann
a9544f5609
lower log level for frequent messages
2023-06-20 15:47:38 +02:00
Jan Eitzinger
d0516f12b0
Merge pull request #154 from ClusterCockpit/hotfix
...
Hotfix
2023-06-20 13:00:00 +02:00
c973a29734
Restore api role access in SecurityCheck
2023-06-20 12:54:26 +02:00
33613cdda0
Sync commit
2023-06-20 10:38:53 +02:00
17113eddca
Fix failed test due to config change
2023-06-15 11:23:50 +02:00
9137931b74
Add cpu_load and mem_used to Job stats
...
Fixes #111
2023-06-15 11:07:48 +02:00
Christoph Kluge
f31d8a3af0
fix test
2023-06-12 11:58:10 +02:00
Christoph Kluge
0d2e20e9e4
Handle users with no roles as "user role"
...
-for backwards compatibility
2023-06-12 11:35:16 +02:00
911dcb6626
Improve Compression Service
...
* Add Timing and more feedback
* Introduce persistent last compressed timestamp
2023-06-10 07:49:02 +02:00
918f1993f2
Add loglevel access. Fix timer labels.
2023-06-09 15:02:22 +02:00
Christoph Kluge
edb1b47281
Add AddJobCount() for analysis view
2023-06-09 13:15:25 +02:00
616095fe66
Add additional job stats, fix test
2023-06-09 11:29:07 +02:00
5ba6f0ed3a
Refactor and adapt to new API
2023-06-09 09:09:41 +02:00
02752f52e4
Merge branch 'master' into refactor-job-repository
2023-06-08 06:22:51 +02:00
fb86ebdbcc
Renaming
2023-06-08 06:18:19 +02:00
Jan Eitzinger
e244a7c85a
Merge pull request #122 from pc2/master
...
Fix database migration order
2023-06-07 17:26:58 +02:00
Jan Eitzinger
1b1e46bf01
Merge pull request #123 from ClusterCockpit/121_fix_filter_reactivity_crash
...
121 fix filter reactivity crash
2023-06-07 17:24:46 +02:00
Michael Schwarz
a183c23e25
Fix truncate call on mysql and provide step 4 migrations for mysql
2023-06-07 16:49:08 +02:00
Christoph Kluge
4e568e60cd
Del JobName from Job type, jobStats name required
...
- Regenerated GQL
- Relates to #121
- return jobStats name as string
2023-06-07 14:13:59 +02:00
becb2bfa3a
Refactor Jobs stats resolver
2023-06-07 11:58:58 +02:00
Michael Schwarz
df240c2394
Fix database migration order
2023-06-07 11:45:58 +02:00
695e8e270a
Split job Repository in multiple files
2023-06-06 10:27:55 +02:00
Jan Eitzinger
797d5da80b
Merge pull request #120 from ClusterCockpit/optimize-sqlite-db-setup
...
Enable SQL hooks for statement debugging
2023-06-05 16:41:52 +02:00
96d036a128
Enable SQL hooks for statement debugging
2023-06-05 16:38:58 +02:00
Jan Eitzinger
e32872181f
Merge pull request #118 from ClusterCockpit/optimize-sqlite-db-setup
...
Add migration for job table contraints
2023-06-05 16:13:24 +02:00
d53ef526dd
Fix tests
2023-06-05 16:10:47 +02:00
e444b83149
Fix errors
2023-06-05 16:07:01 +02:00
5ca4bdbd91
Remove VACUUM
2023-06-05 16:01:14 +02:00
8b6a1c8045
Add migration for job table contraints
2023-06-05 15:58:07 +02:00
Christoph Kluge
a6cb833843
Merge branch 'master' into 97_107_mark_and_show_shared
...
- Solves query.go conflict by splitting QueryJobLinks function aswell
2023-06-01 17:48:43 +02:00
2de85b0120
Add composite indexes
2023-06-01 15:40:37 +02:00
203ccdf73b
Rename test db
2023-06-01 15:29:25 +02:00
447fc574e8
Add query benchmark
2023-06-01 15:27:19 +02:00
Christoph Kluge
37d5db443f
Merge branch 'master' into 97_107_mark_and_show_shared
2023-06-01 15:24:26 +02:00
205a400c6f
Also Benchmarks need _test in name
2023-06-01 14:10:10 +02:00
012b131b23
Refactor for benchmarking
2023-06-01 13:57:35 +02:00
0f87f651f9
Refactor DB connection settings
2023-06-01 08:03:12 +02:00
Christoph Kluge
270750a78d
Move concurrentJobs as field to main job query
2023-05-16 12:42:06 +02:00
19e3ba7290
Improve retention
...
Add db cleanup
Fixes #103
2023-05-11 09:40:13 +02:00
538427d59b
Introduce Retention and compression
2023-05-09 16:33:26 +02:00
788d2cd61c
Fix repo tests
2023-05-05 06:02:17 +02:00
7491ae9e3c
Fix typo in DB schema
2023-05-04 17:26:33 +02:00
066c4553b4
Refactor
2023-05-04 16:27:30 +02:00
5404be3ca7
Add unique constrain on job table
2023-05-04 16:04:57 +02:00
c5b67d64d2
Add Table flush on init
2023-05-04 16:03:04 +02:00
d9b18d1763
Cleanup
...
Use external cc-units
Lint code
Fix import tests
2023-05-04 15:34:36 +02:00
27894ea0a9
Restructure tests
2023-05-04 07:00:30 +02:00
Christoph Kluge
d93610f700
List parallel jobs on node for jobs on shared node
...
- Relates to issue #97
- required GQL schema extension and regeneration
- Works for archived jobs aswell
2023-04-28 12:34:40 +02:00
ebba4371eb
Extract importer package and restructure tests
2023-04-28 08:49:58 +02:00
5a09277d1c
Restructure black box test
...
Remove string ptr from unit prefix
2023-04-25 09:26:48 +02:00
Jan Eitzinger
500ae29d25
Synchronize DB Schema with json schema
...
Rework tests
2023-04-21 12:59:27 +02:00
Jan Eitzinger
10f4e09562
Set all versions and print in version flag
2023-04-12 10:43:46 +02:00
Jan Eitzinger
f8ba79e9e7
Merge branch 'master' into import-data-sanitation
2023-04-07 08:57:42 +02:00
Jan Eitzinger
d420b8b666
Merge branch 'master' into 40_45_82_update_roles
2023-04-07 08:19:04 +02:00
Jan Eitzinger
559445900d
Adapt job iterator
...
* Add option to also load Job Metric Data
* Validate Job Metric Data
* Allow null values in series array
2023-03-31 15:28:35 +02:00
Jan Eitzinger
5d535edb48
Synchronize go job struct with schema
2023-03-29 06:46:33 +02:00
Christoph Kluge
de71fee61e
Merge pull request #98 from giesselmann/fix-updateconfig
...
Fix UpdateConfig
2023-03-21 12:09:40 +01:00
Pay Giesselmann
de822e13fd
Fix UpdateConfig
2023-03-13 10:45:55 +01:00
Christoph Kluge
f37e7c26f6
Rework roles as enum, change AuthSource to enum
2023-03-06 11:44:38 +01:00
Christoph Kluge
4f1f034702
Move ShortRunningJobsDuration value to root config
2023-03-01 16:34:59 +01:00
Christoph Kluge
df44bd9621
Merge branch 'master' into 40_45_82_update_roles
2023-02-28 11:18:46 +01:00
89b1d8f7da
Treat error no change as non fatal
2023-02-25 08:26:56 +01:00
Christoph Kluge
8bd72ce807
Small fixes after full file review
...
- Remove unnecessary field 'project' from GQl and regenerate
- Add newlines to file ends
- Fix command-line manager user addition
2023-02-23 12:33:14 +01:00
Christoph Kluge
d30dba109d
Fix test
2023-02-22 16:57:15 +01:00
Christoph Kluge
365888c645
Fix tag panic and simplify code
2023-02-22 16:49:23 +01:00
Christoph Kluge
d6b8d711f6
Rework searchbar functions, add user jobtable
2023-02-22 16:30:01 +01:00
Christoph Kluge
d0d578a1e6
Fix user.projects up migration file
2023-02-22 10:17:40 +01:00
Christoph Kluge
48728e64c4
Merge branch 'master' into 40_45_82_update_roles
2023-02-22 09:28:41 +01:00
60437ce264
Rebuild GraphQL api and Swagger UI code.
2023-02-21 18:13:16 +01:00
Christoph Kluge
822c8b570c
Remove exists checks from sql migrations to v3
2023-02-21 18:05:29 +01:00
Christoph Kluge
3b7ca17b3c
Merge branch 'master' into 40_45_82_update_roles
2023-02-21 17:36:58 +01:00
Christoph Kluge
acbef39045
Fix header, add migration for user.projects column
2023-02-21 17:33:27 +01:00
af26331f59
Merge back changes from main
2023-02-21 17:18:54 +01:00
Christoph Kluge
e0e51813ad
Merge branch 'master' into 40_45_82_update_roles
2023-02-21 17:17:41 +01:00
Jan Eitzinger
33b20620ee
Merge branch 'master' into sql-repository-opt
2023-02-21 16:21:47 +01:00
999609a17a
Add backend type and adopt cast expressions
...
Fixes #67
2023-02-21 15:22:05 +01:00
63f8555af7
Add migration support for mysql
2023-02-21 14:45:59 +01:00
42d3318125
Add documentation in README
2023-02-21 13:50:15 +01:00
5ff88e1701
Add routine names to sql query timer
2023-02-21 12:08:06 +01:00
a4f5b139f9
Remove hardcoded index generation
2023-02-21 11:36:23 +01:00
8ffb562d6b
Introduce db migration support
2023-02-21 10:57:22 +01:00
643bd3fb21
Add timer calls to repository functions
2023-02-20 15:08:23 +01:00
Christoph Kluge
68efe871c7
Centralize project filter for manager role
...
- Remove all unnecessary frontend code for managerfilters
2023-02-20 11:24:18 +01:00
Christoph Kluge
09bcf9f355
Better implementation for querying multiple users
2023-02-20 10:20:08 +01:00
Christoph Kluge
397ab08b3b
Add support for multiple projects per manager
...
- Handled like roles in admin view
- !! NEW COLUMN CHANGED TO "projects"
2023-02-17 15:45:31 +01:00
Christoph Kluge
bf6ec1bc98
Add search for user names, add names in user table
...
- Regenerated GraphQL due to new required query 'multiUser'
2023-02-17 10:45:27 +01:00
3b38d8042e
Fix tests
2023-02-15 13:07:19 +01:00
033598a656
Remove loglevel notice
2023-02-15 11:50:51 +01:00
491baafd1d
Add index. Formatting
2023-02-15 09:50:27 +01:00
51b2a7f30e
Relocate JobStatistics from GraphQL resolver to JobRepository
2023-02-13 13:53:24 +01:00
aef821fa7e
Add SQL hook for logging and timing
2023-02-13 13:52:17 +01:00
Christoph Kluge
728531946a
Merge branch 'master' into 26_improve_logmessages
2023-02-09 13:52:58 +01:00
Jan Eitzinger
180e9cb09d
Merge pull request #80 from ClusterCockpit/34-52-rework-searchbar
...
Merge Searchbar Rework
2023-02-09 11:38:02 +01:00
Christoph Kluge
ef1b2d7a81
Fix typo
2023-02-01 12:29:58 +01:00
Christoph Kluge
a885e69125
Adapt loglevel for logs, shorten strings, fix formats, streamline
...
- Switched to Warn for most errors, reduces bloat, improves log control
2023-02-01 11:58:27 +01:00
Christoph Kluge
b77bd078e5
Add log messages to error events w/o log message, primaryly error level
...
- "log spam" to be controlled via loglevel flag on startup
2023-01-31 18:28:44 +01:00
Christoph Kluge
7d4f4ab2c8
Merge branch 'master' into 40_45_82_update_roles
2023-01-30 14:33:44 +01:00
Christoph Kluge
186c8f7311
Fix user display configrations not persisting in SQLite table
...
- Was missing user.username key
2023-01-30 14:33:12 +01:00
Christoph Kluge
dd5e0682a1
Fix test file, add suer table project column to test.db
2023-01-30 14:03:44 +01:00
Christoph Kluge
c4d91039e2
Add support role to tag count
...
- works equal to admin
2023-01-30 13:48:27 +01:00
Christoph Kluge
2590ce7e44
Adapt count tags for managers
2023-01-30 13:43:12 +01:00
Christoph Kluge
b2aed2f16b
Add 'project' to user table, add 'manager' role, conditional web render
...
- Addresses issues #40 #45 #82
- Reworked Navigation Header for all roles
- 'Manager' role added, can be assigned a project-id in config by admins
- BREAKING! -> Added 'project' column in SQLite3 table 'user'
- Manager-Assigned project will be added to all graphql filters: Only show Jobs and Users of given project
- 'My Jobs' Tab for all Roles
- Switched from Bool "isAdmin" to integer authLevels
- Removed critical data frontend logging
- Reworked repo.query.SecurityCheck()
2023-01-27 18:36:58 +01:00
Christoph Kluge
834f9d9085
Add role helper functions, add project role barebone, add valid role arr
...
- HasAnyRoles([]string): Checks if user has *one* of the roles
- HasAllRoles([]string): Cheks if user has *all* of the roles
- HasNotRoles([]string): Checks if user has *none* of the roles
- IsValidRole(string): Checks if given string is known valid role
2023-01-25 16:59:16 +01:00
Christoph Kluge
bd770d0e32
Fix return error, fix wrong log path, add notice log where applicable
...
- Adresses issue #26
2023-01-24 12:02:29 +01:00
Christoph Kluge
79a949b55e
Rework pkg/log, add 'loglevel' and 'logdate' flags, streamline
...
- removes some previously added manual location strings: now handled by pkg/log depending on loglevel
- kept manual string locations on fmt print functions
- add 'notice' and 'critical' loglevels
- add 'Panic' and 'Panicf' functions to log panics
- adresses issue #26
2023-01-23 18:48:06 +01:00
Christoph Kluge
24a4244f19
add more information to existing errors logs and panics
...
- '$ROOT/$FILE' for better localization in the code
- add text where none was given
- fix unnecessary sprintf nesting in influxv2 and prometheus metricrepo logging
2023-01-19 16:59:14 +01:00
Christoph Kluge
1873c6a87d
Merge branch 'master' into 34-52-rework-searchbar
2023-01-13 09:58:56 +01:00
Christoph Kluge
7998577076
Remove metadata from scanJob
...
- Leftover from initial jobName query implementation tryouts: unnecessary
2023-01-12 11:57:44 +01:00
Christoph Kluge
84a1c4bdaf
fix wrong vartype for log
2023-01-12 11:34:21 +01:00
Christoph Kluge
bad3f9ea95
Makeover of searchbar, add "not found" results-redirects
...
- Tagged Search has priority
- Best guess search if no tag provided
- "Hard" errors only on searchtag-malformed or searchtag-unknown, or if best guess search fails
2023-01-12 11:26:01 +01:00
