Backend for ClusterCockpit Monitoring Framework. GitHub Mirror
Go to file
Lou Knauer 236f51ba9a Massive speedup in job table initialization
It went from taking like 15 minutes on a ramdisk to taking 430 seconds
on my SSD (~900000 jobs inserted).

- Create indexes after inserts so that they do not need to be updated continually
- Use prepared statements for the job insert
- Bundle 200 job inserts into a single transaction
2021-10-20 09:30:50 +02:00
graph Update GraphQL schema 2021-09-23 11:45:20 +02:00
utils Introduce new golang job-archive backend 2021-03-31 07:23:48 +02:00
.gitignore Introduce clusters query and type 2021-04-22 15:00:54 +02:00
go.mod Optionally initialise new db from JSON files 2021-10-11 11:11:14 +02:00
go.sum Optionally initialise new db from JSON files 2021-10-11 11:11:14 +02:00
gqlgen.yml new filterRanges query and Cluster.filterRanges field 2021-05-06 08:36:24 +02:00
init-db.go Massive speedup in job table initialization 2021-10-20 09:30:50 +02:00
LICENSE Initial checkin. 2019-04-29 10:21:48 +02:00
README.md Change cluster.json to camelCase, go fmt 2021-05-12 18:34:55 +02:00
server.go Optionally initialise new db from JSON files 2021-10-11 11:11:14 +02:00

Run server

  • The server expects the SQLite Job database in ./job.db.
  • The metric data as JSON is expected in ./job-data/<clusterId>/.../data.json.
  • A JSON-description of the clusters is expected in ./job-data/<clusterId>/cluster.json.
  • Run go run server.go
  • The GraphQL backend is located at http://localhost:8080/query .

Debugging and Testing

There is a GraphQL PLayground for testing queries at http://localhost:8080/ .

Example Query:

query($filter: JobFilterList!, $sorting: OrderByInput!, $paging: PageRequest!) {
  jobs(
    filter: $filter
    order: $sorting
    page: $paging
  ) {
    count
    items {
      id
      jobId
      userId
      startTime
      duration
    }
  }
}

Using the Query variables:

{
  "filter": { "list": [
    {"userId": {"contains": "unrz"}},
    {"duration": {"from": 60, "to": 1000}},
    {"startTime": {"from": "2019-06-01T00:00:00.00Z", "to": "2019-10-01T00:00:00.00Z"}}]},
  "sorting": { "field": "start_time", "order": "ASC" },
  "paging": { "itemsPerPage": 20, "page": 1 }
}

Changing the GraphQL schema

  • Edit ./graph/schema.graphqls
  • Regenerate code: gqlgen generate
  • Implement callbacks in graph/resolvers.go