cc-backend/README.md

124 lines
6.6 KiB
Markdown
Raw Normal View History

2022-03-15 17:31:14 +01:00
# ClusterCockpit REST and GraphQL API backend
2021-03-31 08:52:43 +02:00
2022-01-27 09:40:59 +01:00
[![Build](https://github.com/ClusterCockpit/cc-backend/actions/workflows/test.yml/badge.svg)](https://github.com/ClusterCockpit/cc-backend/actions/workflows/test.yml)
2021-03-31 08:52:43 +02:00
2022-03-15 17:31:14 +01:00
This is a Golang backend implementation for a REST and GraphQL API according to the [ClusterCockpit specifications](https://github.com/ClusterCockpit/cc-specifications).
2022-06-23 10:24:36 +02:00
It also includes a web interface for ClusterCockpit.
2022-03-15 17:31:14 +01:00
This implementation replaces the previous PHP Symfony based ClusterCockpit web-interface.
2022-06-23 10:24:36 +02:00
[Here](https://github.com/ClusterCockpit/ClusterCockpit/wiki/Why-we-switched-from-PHP-Symfony-to-a-Golang-based-solution) is a discussion of the reasons why we switched from PHP Symfony to a Golang based solution.
2022-01-07 09:54:12 +01:00
2022-03-15 17:31:14 +01:00
## Overview
This is a golang web backend for the ClusterCockpit job-specific performance monitoring framework.
It provides a REST API for integrating ClusterCockpit with a HPC cluster batch system and external analysis scripts.
Data exchange between the web frontend and backend is based on a GraphQL API.
2022-06-23 10:24:36 +02:00
The web frontend is also served by the backend using [Svelte](https://svelte.dev/) components.
2022-03-15 17:31:14 +01:00
Layout and styling is based on [Bootstrap 5](https://getbootstrap.com/) using [Bootstrap Icons](https://icons.getbootstrap.com/).
2022-06-23 10:24:36 +02:00
The backend uses [SQLite 3](https://sqlite.org/) as relational SQL database by default.
It can optionally use a MySQL/MariaDB database server.
Finished batch jobs are stored in a file-based job archive following [this specification](https://github.com/ClusterCockpit/cc-specifications/tree/master/job-archive).
2022-03-15 17:31:14 +01:00
The backend supports authentication using local accounts or an external LDAP directory.
2022-06-23 10:24:36 +02:00
Authorization for APIs is implemented using [JWT](https://jwt.io/) tokens created with public/private key encryption.
2022-03-15 17:31:14 +01:00
2022-06-24 10:35:54 +02:00
You find more detailed information here:
* `./configs/README.md`: Infos about configuration and setup of cc-backend.
* `./init/README.md`: Infos on how to setup cc-backend as systemd service on Linux.
* `./tools/README.md`: Infos on the JWT authorizatin token workflows in ClusterCockpit.
2022-03-16 10:14:52 +01:00
## Demo Setup
We provide a shell skript that downloads demo data and automatically builds and starts cc-backend.
You need `wget`, `go`, and `yarn` in your path to start the demo. The demo will download 32MB of data (223MB on disk).
```sh
2022-06-23 10:24:36 +02:00
git clone git@github.com:ClusterCockpit/cc-backend.git
2022-07-08 11:01:51 +02:00
cd ./cc-backend
2022-03-16 10:14:52 +01:00
./startDemo.sh
```
2022-06-23 10:24:36 +02:00
You can access the web interface at http://localhost:8080.
Credentials for login: `demo:AdminDev`.
2022-06-24 10:35:54 +02:00
Please note that some views do not work without a metric backend (e.g., the Systems and Status view).
2022-03-16 10:14:52 +01:00
2022-03-15 17:31:14 +01:00
## Howto Build and Run
2021-03-31 08:54:14 +02:00
```sh
2022-06-23 10:24:36 +02:00
git clone git@github.com:ClusterCockpit/cc-backend.git
2021-03-31 08:54:14 +02:00
2022-09-26 14:19:15 +02:00
# Build binary
cd ./cc-backend/
make
# EDIT THE .env FILE BEFORE YOU DEPLOY (Change the secrets)!
# If authentication is disabled, it can be empty.
2022-06-23 10:24:36 +02:00
cp configs/env-template.txt .env
2022-01-17 13:35:49 +01:00
vim ./.env
2022-09-26 15:46:38 +02:00
cp configs/config.json ./
vim ./config.json
2022-09-26 14:19:15 +02:00
#Optional: Link an existing job archive:
ln -s <your-existing-job-archive> ./var/job-archive
# This will first initialize the job.db database by traversing all
# `meta.json` files in the job-archive and add a new user. `--no-server` will cause the
# executable to stop once it has done that instead of starting a server.
2022-09-26 14:19:15 +02:00
./cc-backend --init-db --add-user <your-username>:admin:<your-password>
2022-09-26 14:19:15 +02:00
# Start a HTTP server (HTTPS can be enabled, the default port is 8080).
# The --dev flag enables GraphQL Playground (http://localhost:8080/playground) and Swagger UI (http://localhost:8080/swagger).
./cc-backend --server --dev
# Show other options:
2022-01-27 09:40:59 +01:00
./cc-backend --help
2021-03-31 08:58:34 +02:00
```
2022-03-15 17:31:14 +01:00
### Run as systemd daemon
2022-06-23 10:24:36 +02:00
In order to run this program as a daemon, cc-backend ships with an [example systemd setup](./init/README.md).
2022-03-15 17:31:14 +01:00
## Configuration and Setup
2021-03-31 08:58:34 +02:00
2022-06-23 10:24:36 +02:00
cc-backend can be used as a local web-interface for an existing job archive or as a general web-interface server for a live ClusterCockpit Monitoring framework.
2022-03-15 17:31:14 +01:00
2022-06-23 10:24:36 +02:00
Create your job-archive according to [this specification](https://github.com/ClusterCockpit/cc-specifications/tree/master/job-archive).
At least one cluster with a valid `cluster.json` file is required.
Having no jobs in the job-archive at all is fine.
### Configuration
2022-09-26 14:19:15 +02:00
A config file in the JSON format has to be provided using `--config` to override the defaults.
2022-07-08 11:01:51 +02:00
By default, if there is a `config.json` file in the current directory of the `cc-backend` process, it will be loaded even without the `--config` flag.
2022-06-23 10:24:36 +02:00
You find documentation of all supported configuration and command line options [here](./configs.README.md).
### Update GraphQL schema
2022-06-23 10:24:36 +02:00
This project uses [gqlgen](https://github.com/99designs/gqlgen) for the GraphQL API.
The schema can be found in `./api/schema.graphqls`.
After changing it, you need to run `go run github.com/99designs/gqlgen` which will update `./internal/graph/model`.
In case new resolvers are needed, they will be inserted into `./internal/graph/schema.resolvers.go`, where you will need to implement them.
2022-09-16 06:39:12 +02:00
If you start cc-backend with flag `--dev` the GraphQL Playground UI is available at http://localhost:8080/playground .
### Update Swagger UI
This project integrates [swagger ui](https://swagger.io/tools/swagger-ui/) to document and test its REST API.
The swagger doc files can be found in `./api/`.
You can generate the configuration of swagger-ui by running `go run github.com/swaggo/swag/cmd/swag init -d ./internal/api,./pkg/schema -g rest.go -o ./api `.
You need to move the generated `./api/doc.go` to `./internal/api/doc.go`.
If you start cc-backend with flag `--dev` the Swagger UI is available at http://localhost:8080/swagger .
2019-04-29 10:26:20 +02:00
2022-03-15 17:31:14 +01:00
## Project Structure
2022-06-23 10:24:36 +02:00
- `api/` contains the API schema files for the REST and GraphQL APIs. The REST API is documented in the OpenAPI 3.0 format in [./api/openapi.yaml](./api/openapi.yaml).
- `cmd/cc-backend` contains `main.go` for the main application.
2022-09-16 07:05:44 +02:00
- `cmd/gen-keypair` contains is a small application to generate a compatible JWT keypair includin a README about JWT setup in ClusterCockpit.
2022-06-23 10:24:36 +02:00
- `configs/` contains documentation about configuration and command line options and required environment variables. An example configuration file is provided.
- `init/` contains an example systemd setup for production use.
- `internal/` contains library source code that is not intended to be used by others.
- `pkg/` contains go packages that can also be used by other projects.
- `test/` Test apps and test data.
- `web/` Server side templates and frontend related files:
- `templates` Serverside go templates
- `frontend` Svelte components and static assets for frontend UI
- `gqlgen.yml` configures the behaviour and generation of [gqlgen](https://github.com/99designs/gqlgen).
2022-06-23 10:24:36 +02:00
- `startDemo.sh` is a shell script that sets up demo data, and builds and starts cc-backend.