ClusterCockpit/cc-backend
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-backend synced 2024-12-26 13:29:05 +01:00
Code Issues Packages Releases Activity

Merge pull request #132 from ClusterCockpit/hotfix

Browse Source 
Hotfix
This commit is contained in:
Jan Eitzinger 2023-06-13 15:18:58 +02:00 committed by GitHub
commit 445fe14ac8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 78 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

48
docs/ConfigurationManagement.md
View File

@ -1,37 +1,37 @@
# Release versioning # Release versioning
Releases are numbered with a integer id starting with 1. Releases are numbered with an integer ID, starting with 1.
Every release embeds the following assets into the binary: Each release embeds the following assets in the binary:
* Web-frontend including javascript files and all static assets * Web front-end with Javascript files and all static assets.
* Golang template files for server-side rendering * Golang template files for server-side rendering.
* JSON schema files for validation * JSON schema files for validation.
* Database migration files
Remaining external assets are: The remaining external assets are:
* The SQL database used * The SQL database used
* The job archive * The job archive
* The configuration file `config.json`
Both external assets are also versioned using integer ids. Both external assets are also versioned with integer IDs.
This means every release binary is tied to specific versions for the SQL This means that each release binary is bound to specific versions of the SQL
database and job archive. database and the job archive.
A command line switch `--migrate-db` is provided to migrate the SQL database The configuration file is validated against the current schema on startup.
from a previous to the most recent version. The command line switch `-migrate-db` can be used to upgrade the SQL database
We provide a separate tool `archive-migration` to migrate an existing job to migrate from a previous version to the latest one.
archive from the previous to the most recent version. We offer a separate tool `archive-migration` to migrate an existing job archive
archive from the previous to the latest version.
# Versioning of APIs # Versioning of APIs
cc-backend provides two API backends: cc-backend provides two API backends:
* A REST API for querying jobs * A REST API for querying jobs
* A GraphQL API used for data exchange between web frontend and cc-backend * A GraphQL API for data exchange between web frontend and cc-backend
Both APIs will also be versioned. We still need to decide if we also support Both APIs will also be versioned. We still need to decide wether we will also support
older REST API version using versioning of the endpoint URLs. older REST API version by versioning the endpoint URLs.
# How to build a specific release
# How to migrate the SQL database
# How to migrate the job archive
# How to build
Please always build `cc-backend` with the supplied Makefile. This will ensure
that the frontend is also built correctly and that the version in the binary file is coded
in the binary.

16
docs/adm-customization.md
View File

@ -1,13 +1,13 @@
# Overview # Overview
To customize `cc-backend` means to change the logo and add specific legal texts Customizing `cc-backend` means changing the logo and certain legal texts
in place of the placeholders. To change the logo shown in the navigation bar the instead of the placeholders. To change the logo displayed in the navigation bar, the
file `web/frontend/public/img/logo.png` has to be replaced in the source tree file `web/frontend/public/img/logo.png` in the source tree must be replaced
and cc-backend has to be rebuild. and cc-backend must be rebuild.
# Replace legal texts # Replace legal texts
To replace the legal texts `imprint.tmpl` and `privacy.tmpl` you can place your To replace the `imprint.tmpl` and `privacy.tmpl` legal texts, you can place your
version in `./var/`. On startup `cc-backend` will check if `./var/imprint.tmpl` and/or version in `./var/`. At startup `cc-backend` will check if `./var/imprint.tmpl` and/or
`./var/privacy.tmpl` exist and use those instead of the builtin placeholders. `./var/privacy.tmpl` exist and use them instead of the built-in placeholders.
You may use the placeholders in `web/templates` as blueprint. You can use the placeholders in `web/templates` as a blueprint.

84
docs/adm-upgrade.md
View File

@ -1,33 +1,37 @@
In general an upgrade is not more then replacing the binary. All files that are In general, an upgrade is nothing more than a replacement of the binary file.
required, apart from the database file, the configuration file and the job All the necessary files, except the database file, the configuration file and
archive are embedded into the binary. It is recommended to use a directory where the job archive, are embedded in the binary file. It is recommended to use a
the binary filenames are named using some version tag. This may be, e.g., the date or directory where the file names of the binary files are named with a version
unix epoch time. A symbolic link points to the version to be used. This allows indicator. This can be, for example, the date or the Unix epoch time. A symbolic
to easily switch to previous versions. link points to the version to be used. This makes it easier to switch to earlier
versions.
The database and the job archive are versioned. Every release binary supports The database and the job archive are versioned. Each release binary supports
specific versions of the database and the job archive. In case a version specific versions of the database and job archive. If a version mismatch is
mismatch is detected the application will exit and a migration is required. detected, the application is terminated and migration is required.
**IMPORTANT NOTICE** **IMPORTANT NOTE**
It is recommended to backup the database before any upgrade. It is mandatory It is recommended to make a backup copy of the database before each update. This
to backup in case the database is migrated. Using sqlite this means to stop is mandatory in case the database needs to be migrated. In the case of sqlite,
`cc-backend` and copy the sqlite database file somewhere. this means to stopping `cc-backend` and copying the sqlite database file
somewhere.
# Migrating the database # Migrating the database
After backing up the database execute the the following command to migrate the
databse to the most recent version: After you have backed up the database, run the following command to migrate the
database to the latest version:
``` ```
$ ./cc-backend -migrate-db $ ./cc-backend -migrate-db
``` ```
The migration files are embedded into the binary but can be reviewed in the The migration files are embedded in the binary and can also be viewed in the cc
cc-backend [source tree](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/repository/migrations). backend [source tree](https://github.com/ClusterCockpit/cc-backend/tree/master/internal/repository/migrations).
There are separate migration files for both supported database backends. There are separate migration files for both supported
database backends.
We use the [migrate library](https://github.com/golang-migrate/migrate). We use the [migrate library](https://github.com/golang-migrate/migrate).
In case something goes wrong you can check the state and get the current schema If something goes wrong, you can check the status and get the current schema
(here for sqlite): (here for sqlite):
``` ```
$ sqlite3 var/job.db $ sqlite3 var/job.db
@ -37,38 +41,38 @@ In the sqlite console execute:
.schema .schema
``` ```
to get the current databse schema. to get the current databse schema.
You can query the current version and if the migration failed with: You can query the current version and whether the migration failed with:
``` ```
SELECT * FROM schema_migrations; SELECT * FROM schema_migrations;
``` ```
The first column indicates the current database version and the second column is The first column indicates the current database version and the second column is
a dirty flag indicating if the migration was successful. a dirty flag indicating whether the migration was successful.
# Migrating the job archive # Migrating the job archive
To migrate the job archive a separate tool (`archive-migration`) is required that is part of the Job archive migration requires a separate tool (`archive-migration`), which is
`cc-backend` source tree (build with `go build ./tools/archive-migration`) and also provided as part of releases. part of the cc-backend source tree (build with `go build ./tools/archive-migration`)
and is also provided as part of the releases.
Migration is only supported between two subsequent versions. The migration tool Migration is supported only between two successive releases. The migration tool
will migrate the existing job archive into a new job archive. This means there migrates the existing job archive to a new job archive. This means that there
has to be enough disk space for two complete job-archives. If the tool is must be enough disk space for two complete job archives. If the tool is called
called without options: without options:
``` ```
$ ./archive-migration $ ./archive-migration
``` ```
it is assumed that there is a job archive in `./var/job-archive`. The new job it is assumed that a job archive exists in `./var/job-archive`. The new job
archive will be written to `./var/job-archive-new`. Because execution is archive is written to `./var/job-archive-new`. Since execution is threaded in case
threaded in case of a fatal error it is impossible to pinpoint in which job the of a fatal error, it is impossible to determine in which job the error occurred.
error occured. In this case you can run the tool in debug mode (using the flag In this case, you can run the tool in debug mode (with the `-debug` flag). In
`-debug`). In debug mode threading will be disabled and the job id of every debug mode, threading is disabled and the job ID of each migrated job is output.
migrated job is printed. Jobs with empty files are skipped. Between multiple runs Jobs with empty files will be skipped. Between multiple runs of the tools, the
of the tools the directory `job-archive-new` has to be moved or deleted. `job-archive-new` directory must be moved or deleted.
The cluster.json files in `job-archive-new` need to be reviewed for errors, in The `cluster.json` files in `job-archive-new` must be checked for errors, especially
particular it has to be checked if the aggregation attribute is set correctly whether the aggregation attribute is set correctly for all metrics.
for all metrics.
The migration will take in the order of hours for fairly large (several hundred Migration takes several hours for relatively large job archives (several hundred
GB) job archives. A versioned job archive contains a file `version.txt` in the GB). A versioned job archive contains a version.txt file in the root directory
job archive root directory. This file contains the version as unsigned integer. of the job archive. This file contains the version as an unsigned integer.

7
web/frontend/README.md
View File

@ -12,18 +12,15 @@ Builds on:
## Get started ## Get started
[Yarn](https://yarnpkg.com/) is recommended for package management.
Due to an issue with Yarn v2 you have to stick to Yarn v1.
Install the dependencies... Install the dependencies...
```bash ```bash
yarn install npm install
``` ```
...then build using [Rollup](https://rollupjs.org): ...then build using [Rollup](https://rollupjs.org):
```bash ```bash
yarn build npm run build
``` ```