diff --git a/.gitignore b/.gitignore index e1a840e..458c508 100644 --- a/.gitignore +++ b/.gitignore @@ -1,9 +1,6 @@ -data/job-archive -data/job-archive/** -data/sqldata -data/cc-metric-store -data/cc-metric-store-source -data/slurm +# everything under data/ is generated by dataGenerationScript.sh +data/ + cc-backend/** -.vscode cc-backend +.vscode diff --git a/README.md b/README.md index 7dfa821..4ef8761 100755 --- a/README.md +++ b/README.md @@ -70,7 +70,8 @@ If you are using different Linux flavors, you will have to adapt to start `cc-backend`. 1. By default, you can access `cc-backend` in your browser at - `http://localhost:8080`. You can shut down the cc-backend server by pressing + `http://localhost:8088` (port 8080 is taken by the KeyCloak container). You + can shut down the cc-backend server by pressing `CTRL-C`, remember to also shut down all containers via `$> docker-compose down` afterwards. @@ -83,24 +84,127 @@ Credentials for the preconfigured demo user are: - User: `demo` - Password: `demo` -Credentials for the preconfigured LDAP user are: +The LDAP user directory is defined in `./data/ldap/add_users.ldif`. Every +account's password equals its user name: -- User: `ldapuser` -- Password: `ldapuser` +| User | Group | Role in ClusterCockpit | +| ----------- | ------------ | ---------------------- | +| `ldapuser` | – | `user` | +| `ccuser` | – | `user` | +| `ccadmin` | `cc-admin` | `user`, `admin` | +| `ccsupport` | `cc-support` | `user` | +| `ccmanager` | `cc-manager` | `user` | +| `ccapi` | `cc-api` | `user` | -You can also login as regular user using any credential in the LDAP user -directory at `./data/ldap/users.ldif`. +### LDAP role sync + +cc-backend can derive elevated roles from LDAP group membership via +`auth.ldap.role-filters` in `config.json`. Each entry maps a role to an LDAP +filter that is evaluated against the *user* entry; the preconfigured setup maps +the `cc-admin` group to the `admin` role: + +```json +"role-filters": { + "admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)" +} +``` + +The `memberOf` attribute is maintained automatically by the `memberof` overlay +of the openldap image, which is configured for `groupOfUniqueNames` / +`uniqueMember` — the group entries in the ldif use those classes accordingly. + +LDAP is authoritative for every role listed in `role-filters`: a role is granted +when the filter matches and revoked when it no longer does. Roles that are not +listed are never touched. To exercise the other groups, add them as well: + +```json +"role-filters": { + "admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)", + "support": "(memberOf=cn=cc-support,ou=groups,dc=example,dc=com)", + "manager": "(memberOf=cn=cc-manager,ou=groups,dc=example,dc=com)", + "api": "(memberOf=cn=cc-api,ou=groups,dc=example,dc=com)" +} +``` + +Roles are applied when the account is first synced (`sync-user-on-login`) and +refreshed on every subsequent login (`update-user-on-login`), as well as by the +periodic sync task (`sync-interval`). + +> Note: The ldif is only applied when the openldap container initialises its +> database. After changing it, recreate the container and its volumes with +> `docker compose rm -sfv openldap`, or load the changes into the running +> directory manually with `ldapadd`. + +### OIDC login via KeyCloak + +The KeyCloak container imports the realm `clustercockpit` from +`./keycloak/import/clustercockpit-realm.json` on first start. It defines the +confidential client `cc-backend`, the realm roles `cc-admin`, `cc-support`, +`cc-manager` and `cc-api`, and these accounts (password equals user name): + +| User | Realm role | Role in ClusterCockpit | +| ----------- | ------------ | ---------------------- | +| `kcuser` | – | `user` | +| `kcadmin` | `cc-admin` | `admin` | +| `kcsupport` | `cc-support` | `user` | +| `kcmanager` | `cc-manager` | `user` | +| `kcapi` | `cc-api` | `user` | + +The KeyCloak admin console is at `http://localhost:8080` (`admin` / `admin`). + +The `auth.oidc.role-mapping` section of `config.json` translates realm roles +into ClusterCockpit roles. The preconfigured setup maps `cc-admin` to `admin`: + +```json +"oidc": { + "provider": "http://localhost:8080/realms/clustercockpit", + "client-id": "cc-backend", + "client-secret": "cc-backend-dev-secret", + "sync-user-on-login": true, + "update-user-on-login": true, + "role-mapping": { + "cc-admin": "admin" + } +} +``` + +Unlike the LDAP role filters, `role-mapping` is the *sole* source of roles for +OIDC logins: a role in the token grants a ClusterCockpit role only if it is +listed here, and unmapped roles are ignored — literal role names such as `admin` +would also have to be mapped explicitly. Accounts without a mapped role get +`user`. Add the remaining groups to exercise them: + +```json +"role-mapping": { + "cc-admin": "admin", + "cc-support": "support", + "cc-manager": "manager", + "cc-api": "api" +} +``` + +Roles are read from the `realm_access.roles` claim of the **ID token**. The +built-in KeyCloak `roles` client scope only adds that claim to the *access* +token, so the imported client carries its own `realm roles in id token` +protocol mapper with `id.token.claim` enabled. Keep that mapper when editing +the realm, otherwise every OIDC login falls back to the plain `user` role. + +> Note: `cc-backend` resolves the OIDC provider at startup and aborts if it is +> unreachable, so the KeyCloak container has to be up before starting the +> server. The realm is only imported while the KeyCloak database is empty — +> to re-import after editing the realm file, recreate KeyCloak *and* its +> database with `docker compose rm -sf keycloak postgres`. ## Preconfigured setup between docker services and ClusterCockpit components When you are done cloning the cc-backend repo and once you execute `setupDev.sh` file, it will copy a preconfigured `config.json` from `misc/config.json` and replace the `cc-backend/config.json`, which will be used by cc-backend, once you start the server. The preconfigured config.json attaches to: -#### 1. OpenLDAP docker service on port 389 +### 1. OpenLDAP docker service on port 389 -#### 2. cc-metric-store docker service on port 8084 +### 2. cc-metric-store docker service on port 8084 -#### 3. cc-slurm-adapter is running on slurmctld docker service +### 3. cc-slurm-adapter is running on slurmctld docker service cc-metric-store also has a preconfigured `config.json` in `cc-metric-store/config.json` which attaches to NATS docker service on port 4222 @@ -194,7 +298,7 @@ custom CURL commands. ## Known Issues - `docker-compose` installed on Ubuntu (18.04, 20.04) via `apt-get` can not correctly parse `docker-compose.yml` due to version differences. Install latest version of `docker-compose` from instead. -- You need to ensure that no other web server is running on ports 8080 (cc-backend), 8084 (cc-metric-store), 4222 and 8222 (Nats). If one or more ports are already in use, you have to adapt the related config accordingly. +- You need to ensure that no other web server is running on ports 8088 (cc-backend), 8080 (KeyCloak), 8084 (cc-metric-store), 4222 and 8222 (Nats). If one or more ports are already in use, you have to adapt the related config accordingly. - Existing VPN connections sometimes cause problems with docker. If `docker-compose` does not start up correctly, try disabling any active VPN connection. Refer to for further information. ## Docker services and restarting the services diff --git a/cc-metric-store/Dockerfile b/cc-metric-store/Dockerfile index e7e6d1d..371536e 100644 --- a/cc-metric-store/Dockerfile +++ b/cc-metric-store/Dockerfile @@ -1,4 +1,4 @@ -FROM golang:1.22.4 +FROM golang:1.26.4 RUN apt-get update RUN apt-get -y install git diff --git a/data/ldap/bootstrap.ldif b/data/ldap/bootstrap.ldif deleted file mode 100644 index e9484a6..0000000 --- a/data/ldap/bootstrap.ldif +++ /dev/null @@ -1,239 +0,0 @@ -# ClusterCockpit Bootstrap LDAP Directory -# ========================================= -# Domain: dc=example,dc=com (LDAP_DOMAIN=example.com in docker-compose.yml) -# Admin DN: cn=admin,dc=example,dc=com (set via LDAP_ADMIN_PASSWORD env) -# -# All test user passwords: "password" -# {SHA} hash verification: slappasswd -h {SHA} -s password -# -# Suggested cc-backend ldap config (config.json): -# "url": "ldap://ldap:389" -# "user-base": "ou=people,dc=example,dc=com" -# "search-dn": "uid=ccbinduser,ou=people,dc=example,dc=com" -# "user-bind": "uid={username},ou=people,dc=example,dc=com" -# "user-filter": "(&(objectclass=posixAccount)(!(uid=ccbinduser)))" -# "username-attr": "gecos" -# "uid-attr": "uid" -# "sync-password": "password" -# -# ClusterCockpit roles (from cc-lib/schema/user.go): -# anonymous < api < user < manager < support < admin -# ========================================= - -# --------------------------------------------------------------------------- -# Organizational Units -# --------------------------------------------------------------------------- - -dn: ou=people,dc=example,dc=com -objectClass: organizationalUnit -objectClass: top -ou: people -description: HPC user accounts - -dn: ou=groups,dc=example,dc=com -objectClass: organizationalUnit -objectClass: top -ou: groups -description: HPC project groups and ClusterCockpit role groups - -# --------------------------------------------------------------------------- -# Service account used by cc-backend for LDAP search binding -# --------------------------------------------------------------------------- - -dn: uid=ccbinduser,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: CC Bind User -sn: BindUser -uid: ccbinduser -uidNumber: 500 -gidNumber: 500 -homeDirectory: /home/ccbinduser -description: Service account for cc-backend LDAP search -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# --------------------------------------------------------------------------- -# Test users -# Role membership is tracked via cc-role-* groups below. -# --------------------------------------------------------------------------- - -# admin01 — ClusterCockpit admin -dn: uid=admin01,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Admin User -sn: User -uid: admin01 -uidNumber: 1001 -gidNumber: 1001 -homeDirectory: /home/admin01 -gecos: Admin User -mail: admin01@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# support01 — ClusterCockpit support staff -dn: uid=support01,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Support User -sn: User -uid: support01 -uidNumber: 1002 -gidNumber: 1001 -homeDirectory: /home/support01 -gecos: Support User -mail: support01@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# manager01 — ClusterCockpit project manager -dn: uid=manager01,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Manager User -sn: User -uid: manager01 -uidNumber: 1003 -gidNumber: 1001 -homeDirectory: /home/manager01 -gecos: Manager User -mail: manager01@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# user01 — regular HPC user -dn: uid=user01,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Regular User 01 -sn: User -uid: user01 -uidNumber: 1010 -gidNumber: 1001 -homeDirectory: /home/user01 -gecos: Regular User 01 -mail: user01@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# user02 — regular HPC user (also member of a project group) -dn: uid=user02,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Regular User 02 -sn: User -uid: user02 -uidNumber: 1011 -gidNumber: 1001 -homeDirectory: /home/user02 -gecos: Regular User 02 -mail: user02@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# user03 — regular HPC user (also member of a project group) -dn: uid=user03,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: Regular User 03 -sn: User -uid: user03 -uidNumber: 1012 -gidNumber: 1001 -homeDirectory: /home/user03 -gecos: Regular User 03 -mail: user03@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# apiuser01 — programmatic/service API access -dn: uid=apiuser01,ou=people,dc=example,dc=com -objectClass: inetOrgPerson -objectClass: posixAccount -objectClass: top -cn: API User 01 -sn: User -uid: apiuser01 -uidNumber: 1020 -gidNumber: 1001 -homeDirectory: /home/apiuser01 -gecos: API User 01 -mail: apiuser01@example.com -userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g= - -# --------------------------------------------------------------------------- -# ClusterCockpit role groups -# These map to cc-lib Role constants: admin, support, manager, user, api -# cc-backend can use these for group-based user filtering or future role sync. -# Example user-filter to restrict login to group members: -# (&(objectclass=posixAccount)(memberOf=cn=cc-users,ou=groups,dc=example,dc=com)) -# Note: memberOf requires the memberof overlay; use memberUid for posixGroup. -# --------------------------------------------------------------------------- - -dn: cn=cc-admins,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: cc-admins -gidNumber: 2000 -description: ClusterCockpit administrators (role: admin) -memberUid: admin01 - -dn: cn=cc-support,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: cc-support -gidNumber: 2001 -description: ClusterCockpit support staff (role: support) -memberUid: support01 - -dn: cn=cc-managers,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: cc-managers -gidNumber: 2002 -description: ClusterCockpit project managers (role: manager) -memberUid: manager01 - -dn: cn=cc-users,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: cc-users -gidNumber: 2003 -description: ClusterCockpit regular users (role: user) -memberUid: user01 -memberUid: user02 -memberUid: user03 - -dn: cn=cc-api,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: cc-api -gidNumber: 2004 -description: ClusterCockpit API/service accounts (role: api) -memberUid: apiuser01 - -# --------------------------------------------------------------------------- -# HPC project groups (for testing manager project-scoping) -# A manager assigned to project hpc_proj_alpha can view all jobs in that project. -# --------------------------------------------------------------------------- - -dn: cn=hpc_proj_alpha,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: hpc_proj_alpha -gidNumber: 3001 -description: HPC project alpha -memberUid: manager01 -memberUid: user01 -memberUid: user02 - -dn: cn=hpc_proj_beta,ou=groups,dc=example,dc=com -objectClass: posixGroup -objectClass: top -cn: hpc_proj_beta -gidNumber: 3002 -description: HPC project beta -memberUid: manager01 -memberUid: user03 diff --git a/dataGenerationScript.sh b/dataGenerationScript.sh index fac98a2..0ae5eff 100755 --- a/dataGenerationScript.sh +++ b/dataGenerationScript.sh @@ -37,16 +37,44 @@ else echo "'data/mariadb' already exists!" fi -# A simple configuration file for openldap docker service. -# Creates a simple user 'ldapuser' with password 'ldapuser'. -# This file automatically picked by openldap after the docker service starts. +# Configuration file for the openldap docker service. +# Creates the dev users (password == uid) and the cc-* role groups used by +# auth.ldap.role-filters in config.json. +# This file is automatically picked up by openldap on first start. if [ ! -d data/ldap ]; then mkdir -p data/ldap - cat >data/ldap/add_users.ldif <data/ldap/add_users.ldif <<'LDIF' +# ClusterCockpit dev LDAP directory +# ================================= +# Applied by the osixia/openldap container on first init via +# /container/service/slapd/assets/config/bootstrap/ldif/custom. +# +# Every account's password equals its uid (e.g. ccadmin / ccadmin). +# Hashes are {SSHA}; regenerate with `slappasswd -h {SSHA} -s `. +# +# Role sync (auth.ldap.role-filters in config.json) matches on the memberOf +# attribute of the *user* entry. The memberof overlay of this image maintains +# memberOf for groupOfUniqueNames/uniqueMember, so the group entries below must +# come after the user entries they reference. + +# --------------------------------------------------------------------------- +# Organizational units +# --------------------------------------------------------------------------- + dn: ou=users,dc=example,dc=com objectClass: organizationalUnit +objectClass: top ou: users +dn: ou=groups,dc=example,dc=com +objectClass: organizationalUnit +objectClass: top +ou: groups + +# --------------------------------------------------------------------------- +# Users +# --------------------------------------------------------------------------- + dn: uid=ldapuser,ou=users,dc=example,dc=com objectClass: inetOrgPerson objectClass: posixAccount @@ -55,10 +83,104 @@ cn: Ldap User sn: User uid: ldapuser uidNumber: 1 -gidNumber: 1 +gidNumber: 100 homeDirectory: /home/ldapuser -userPassword: {SSHA}sQRqFQtuiupej7J/rbrQrTwYEHDduV+N -EOF +userPassword: {SSHA}0OhsiEb+IXanz71bd7jyFhvy0xhPJ34e + +dn: uid=ccadmin,ou=users,dc=example,dc=com +objectClass: inetOrgPerson +objectClass: posixAccount +objectClass: top +cn: CC Admin +sn: Admin +uid: ccadmin +uidNumber: 2000 +gidNumber: 100 +homeDirectory: /home/ccadmin +userPassword: {SSHA}HfIVI9GeNot9ea5bSXgKNcF6kP0MUyfv + +dn: uid=ccsupport,ou=users,dc=example,dc=com +objectClass: inetOrgPerson +objectClass: posixAccount +objectClass: top +cn: CC Support +sn: Support +uid: ccsupport +uidNumber: 2001 +gidNumber: 100 +homeDirectory: /home/ccsupport +userPassword: {SSHA}7zCZNDYQZPZl8jcBSkU+UnmxCR8lD9sm + +dn: uid=ccmanager,ou=users,dc=example,dc=com +objectClass: inetOrgPerson +objectClass: posixAccount +objectClass: top +cn: CC Manager +sn: Manager +uid: ccmanager +uidNumber: 2002 +gidNumber: 100 +homeDirectory: /home/ccmanager +userPassword: {SSHA}DKcUb2h8DWbDtTjXtIvwDi4Bf29vzhKK + +dn: uid=ccapi,ou=users,dc=example,dc=com +objectClass: inetOrgPerson +objectClass: posixAccount +objectClass: top +cn: CC Api +sn: Api +uid: ccapi +uidNumber: 2003 +gidNumber: 100 +homeDirectory: /home/ccapi +userPassword: {SSHA}dhcxIxs883hX2CTpkQp7cNgc0+puJ23z + +dn: uid=ccuser,ou=users,dc=example,dc=com +objectClass: inetOrgPerson +objectClass: posixAccount +objectClass: top +cn: CC User +sn: User +uid: ccuser +uidNumber: 2004 +gidNumber: 100 +homeDirectory: /home/ccuser +userPassword: {SSHA}UKYfiDmhT/H01R20rtYYOBj7t/w7yLyW + +# --------------------------------------------------------------------------- +# Role groups +# The memberof overlay of this image is configured for groupOfUniqueNames / +# uniqueMember, so these classes are required for memberOf to be maintained. +# --------------------------------------------------------------------------- + +dn: cn=cc-admin,ou=groups,dc=example,dc=com +objectClass: groupOfUniqueNames +objectClass: top +cn: cc-admin +description: Grants the ClusterCockpit admin role +uniqueMember: uid=ccadmin,ou=users,dc=example,dc=com + +dn: cn=cc-support,ou=groups,dc=example,dc=com +objectClass: groupOfUniqueNames +objectClass: top +cn: cc-support +description: Grants the ClusterCockpit support role +uniqueMember: uid=ccsupport,ou=users,dc=example,dc=com + +dn: cn=cc-manager,ou=groups,dc=example,dc=com +objectClass: groupOfUniqueNames +objectClass: top +cn: cc-manager +description: Grants the ClusterCockpit manager role +uniqueMember: uid=ccmanager,ou=users,dc=example,dc=com + +dn: cn=cc-api,ou=groups,dc=example,dc=com +objectClass: groupOfUniqueNames +objectClass: top +cn: cc-api +description: Grants the ClusterCockpit api role +uniqueMember: uid=ccapi,ou=users,dc=example,dc=com +LDIF else echo "'data/ldap' already exists!" diff --git a/docker-compose.yml b/docker-compose.yml index 8269c8d..1396af3 100755 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -57,7 +57,11 @@ services: ports: - "0.0.0.0:8080:8080" restart: always - command: --verbose start --optimized + depends_on: + - postgres + volumes: + - ./keycloak/import:/opt/keycloak/data/import:ro + command: --verbose start --optimized --import-realm mariadb: container_name: mariadb diff --git a/keycloak/import/clustercockpit-realm.json b/keycloak/import/clustercockpit-realm.json new file mode 100644 index 0000000..febc945 --- /dev/null +++ b/keycloak/import/clustercockpit-realm.json @@ -0,0 +1,111 @@ +{ + "realm": "clustercockpit", + "displayName": "ClusterCockpit", + "enabled": true, + "sslRequired": "none", + "registrationAllowed": false, + "loginWithEmailAllowed": true, + "roles": { + "realm": [ + { "name": "cc-admin", "description": "Grants the ClusterCockpit admin role" }, + { "name": "cc-support", "description": "Grants the ClusterCockpit support role" }, + { "name": "cc-manager", "description": "Grants the ClusterCockpit manager role" }, + { "name": "cc-api", "description": "Grants the ClusterCockpit api role" } + ] + }, + "clients": [ + { + "clientId": "cc-backend", + "name": "ClusterCockpit Backend", + "description": "Confidential OIDC client used by cc-backend", + "enabled": true, + "protocol": "openid-connect", + "publicClient": false, + "secret": "cc-backend-dev-secret", + "standardFlowEnabled": true, + "directAccessGrantsEnabled": true, + "implicitFlowEnabled": false, + "serviceAccountsEnabled": false, + "redirectUris": [ + "http://localhost:8088/oidc-callback", + "http://127.0.0.1:8088/oidc-callback" + ], + "webOrigins": [ + "http://localhost:8088", + "http://127.0.0.1:8088" + ], + "attributes": { + "pkce.code.challenge.method": "S256", + "post.logout.redirect.uris": "http://localhost:8088/*" + }, + "protocolMappers": [ + { + "name": "realm roles in id token", + "protocol": "openid-connect", + "protocolMapper": "oidc-usermodel-realm-role-mapper", + "consentRequired": false, + "config": { + "multivalued": "true", + "claim.name": "realm_access.roles", + "jsonType.label": "String", + "id.token.claim": "true", + "access.token.claim": "true", + "userinfo.token.claim": "false" + } + } + ] + } + ], + "users": [ + { + "username": "kcadmin", + "enabled": true, + "emailVerified": true, + "firstName": "KC", + "lastName": "Admin", + "email": "kcadmin@example.com", + "credentials": [{ "type": "password", "value": "kcadmin", "temporary": false }], + "realmRoles": ["default-roles-clustercockpit", "cc-admin"] + }, + { + "username": "kcsupport", + "enabled": true, + "emailVerified": true, + "firstName": "KC", + "lastName": "Support", + "email": "kcsupport@example.com", + "credentials": [{ "type": "password", "value": "kcsupport", "temporary": false }], + "realmRoles": ["default-roles-clustercockpit", "cc-support"] + }, + { + "username": "kcmanager", + "enabled": true, + "emailVerified": true, + "firstName": "KC", + "lastName": "Manager", + "email": "kcmanager@example.com", + "credentials": [{ "type": "password", "value": "kcmanager", "temporary": false }], + "realmRoles": ["default-roles-clustercockpit", "cc-manager"] + }, + { + "username": "kcapi", + "enabled": true, + "emailVerified": true, + "firstName": "KC", + "lastName": "Api", + "email": "kcapi@example.com", + "credentials": [{ "type": "password", "value": "kcapi", "temporary": false }], + "realmRoles": ["default-roles-clustercockpit", "cc-api"] + }, + { + "username": "kcuser", + "enabled": true, + "emailVerified": true, + "firstName": "KC", + "lastName": "User", + "email": "kcuser@example.com", + "credentials": [{ "type": "password", "value": "kcuser", "temporary": false }], + "realmRoles": ["default-roles-clustercockpit"] + } + ] +} diff --git a/misc/config.json b/misc/config.json index dcb1745..153d947 100644 --- a/misc/config.json +++ b/misc/config.json @@ -1,78 +1,84 @@ { - "addr": "127.0.0.1:8080", - "short-running-jobs-duration": 300, + "main": { + "addr": "0.0.0.0:8088", + "short-running-jobs-duration": 300, + "emission-constant": 317, + "resampling": { + "minimum-points": 600, + "trigger": 30, + "resolutions": [ + 600, + 300, + 120, + 60 + ] + }, + "api-subjects": { + "subject-job-event": "cc.job.event", + "subject-node-state": "cc.node.state" + } + }, + "nats": { + "address": "nats://0.0.0.0:4222", + "username": "root", + "password": "root" + }, + "auth": { + "jwts": { + "max-age": "2000h", + "public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0=", + "private-key": "dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q==" + }, + "oidc": { + "provider": "http://localhost:8080/realms/clustercockpit", + "client-id": "cc-backend", + "client-secret": "cc-backend-dev-secret", + "sync-user-on-login": true, + "update-user-on-login": true, + "role-mapping": { + "cc-admin": "admin" + } + }, + "ldap": { + "url": "ldap://ldap:389", + "user-base": "ou=users,dc=example,dc=com", + "search-dn": "cn=admin,dc=example,dc=com", + "user-bind": "uid={username},ou=users,dc=example,dc=com", + "user-filter": "(objectclass=posixAccount)", + "username-attr": "cn", + "uid-attr": "uid", + "sync-password": "mashup", + "sync-interval": "2h", + "sync-user-on-login": true, + "update-user-on-login": true, + "role-filters": { + "admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)" + } + } + }, + "cron": { + "commit-job-worker": "1m", + "duration-worker": "5m", + "footprint-worker": "10m" + }, "archive": { "kind": "file", "path": "./var/job-archive" }, - "jwts": { - "max-age": "2000h" - }, - "apiAllowedIPs": [ - "*" - ], - "ldap": { - "url": "ldap://0.0.0.0", - "user_base": "ou=users,dc=example,dc=com", - "search_dn": "cn=admin,dc=example,dc=com", - "user_bind": "uid={username},ou=users,dc=example,dc=com", - "user_filter": "(&(objectclass=posixAccount))", - "syncUserOnLogin": true - }, - "enable-resampling": { - "trigger": 30, - "resolutions": [ - 600, - 300, - 120, - 60 - ] - }, - "emission-constant": 317, - "clusters": [ + "metric-store-external": [ { - "name": "fritz", - "metricDataRepository": { - "kind": "cc-metric-store", - "url": "http://0.0.0.0:8084", - "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw" - }, - "filterRanges": { - "numNodes": { - "from": 1, - "to": 64 - }, - "duration": { - "from": 0, - "to": 86400 - }, - "startTime": { - "from": "2022-01-01T00:00:00Z", - "to": null - } - } + "scope": "fritz", + "url": "http://0.0.0.0:8084", + "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw" }, { - "name": "alex", - "metricDataRepository": { - "kind": "cc-metric-store", - "url": "http://0.0.0.0:8084", - "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw" - }, - "filterRanges": { - "numNodes": { - "from": 1, - "to": 64 - }, - "duration": { - "from": 0, - "to": 86400 - }, - "startTime": { - "from": "2022-01-01T00:00:00Z", - "to": null - } - } + "scope": "alex", + "url": "http://0.0.0.0:8084", + "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw" } - ] -} \ No newline at end of file + ], + "metric-store": { + "retention-in-memory": "24h", + "memory-cap": 100 + } +} diff --git a/resetDev.sh b/resetDev.sh new file mode 100755 index 0000000..eb89244 --- /dev/null +++ b/resetDev.sh @@ -0,0 +1,58 @@ +#!/bin/bash +set -euo pipefail + +echo "" +echo "|--------------------------------------------------------------------------------------|" +echo "| cc-docker dev environment reset script |" +echo "| This will stop all containers, remove volumes, and delete all generated data. |" +echo "| After completion, re-run ./setupDev.sh to reinitialize. |" +echo "|--------------------------------------------------------------------------------------|" +echo "" + +read -p "This will delete all generated data and cannot be undone. Continue? [y/N] " -r +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Aborted." + exit 0 +fi + +DOCKER_COMPOSE="" + +if docker-compose --version &>/dev/null 2>&1; then + DOCKER_COMPOSE="docker-compose" +fi + +if docker compose version &>/dev/null 2>&1; then + DOCKER_COMPOSE="docker compose" +fi + +if [[ -z "${DOCKER_COMPOSE}" ]]; then + echo "docker-compose not found!" + exit 1 +fi + +echo "Stopping containers and removing volumes..." +$DOCKER_COMPOSE down --volumes --remove-orphans + +echo "Removing data directory..." +if [ -d data ]; then + rm -rf data +fi + +echo "Removing cc-backend generated files..." +if [ -d cc-backend/var ]; then + rm -rf cc-backend/var +fi +if [ -f cc-backend/.env ]; then + rm cc-backend/.env +fi +if [ -f cc-backend/config.json ]; then + rm cc-backend/config.json +fi + +echo "" +echo "|--------------------------------------------------------------------------------------|" +echo "| Reset complete. |" +echo "| Run ./setupDev.sh to reinitialize the dev environment. |" +echo "| To also remove built Docker images, run: docker-compose down --rmi all |" +echo "|--------------------------------------------------------------------------------------|" +echo "" diff --git a/setupDev.sh b/setupDev.sh index 48ca516..56917e9 100755 --- a/setupDev.sh +++ b/setupDev.sh @@ -65,12 +65,8 @@ if [ ! -d var ]; then wget https://hpc-mover.rrze.uni-erlangen.de/HPC-Data/0x7b58aefb/eig7ahyo6fo2bais0ephuf2aitohv1ai/job-archive-demo.tar tar xf job-archive-demo.tar rm ./job-archive-demo.tar - - cp ./configs/env-template.txt .env cp ../misc/config.json config.json - sed -i 's/"addr": *"127\.0\.0\.1:8080"/"addr": "0.0.0.0:8080"/' config.json - make ./cc-backend -migrate-db @@ -113,7 +109,6 @@ cd ../.. $DOCKER_COMPOSE build $DOCKER_COMPOSE up -d - echo "" echo "|--------------------------------------------------------------------------------------|" echo "| Check logs for each slurm service by using these commands: |" diff --git a/slurm/README.md b/slurm/README.md index 6c6d2d9..33db4af 100644 --- a/slurm/README.md +++ b/slurm/README.md @@ -4,15 +4,11 @@ Use [Docker](https://www.docker.com/) to explore the various components of [Slurm](https://www.schedmd.com/index.php) -This work represents a small exploratory Slurm cluster using CentOS 7 based Docker images. The intent was to learn the basics of Slurm prior to extending the concept to a more distributed environment. +This work represents a small exploratory Slurm cluster using AlmaLinux 9 based Docker images. Images include: -- [Slurm 19.05.1](https://slurm.schedmd.com) - installed from [rpm packages](packages) -- [OpenMPI 3.0.1](https://www.open-mpi.org/doc/current/) - installed from [rpm packages](packages) -- [Lmod 7.7](http://lmod.readthedocs.io/en/latest/index.html) - installed from [distribution files](https://sourceforge.net/projects/lmod/files/) - - [Lmod module packages for CentOS 7](https://github.com/scidas/lmod-modules-centos) - Organized for Slurm-in-Docker use - - [Using Lmod with Slurm-in-Docker](using-lmod-with-slurm-in-docker.md) documentation +- [Slurm 24.05.3](https://slurm.schedmd.com) - built from source RPMs inside the container ## Contents @@ -63,33 +59,24 @@ TODO: Have software check validity of custom configuration files. ## Build -Build the slurm RPM files by following the instructions in the [packages](packages) directory. - **Create the base Slurm image**: -Copy the `packages/centos-7/rpms` directory to the `base` directory - ``` cd base/ -cp -r ../packages/centos-7/rpms . +make ``` -Build the base image - -``` -docker build -t scidas/slurm.base:19.05.1 . -``` +This builds `clustercockpit/slurm.base:24.05.3` from AlmaLinux 9, compiling Slurm RPMs from source with JWT and slurmrestd support. Verify image build ```console $ docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -scidas/slurm.base 19.05.1 1600621cb483 Less than a second ago 819MB -... +REPOSITORY TAG IMAGE ID CREATED SIZE +clustercockpit/slurm.base 24.05.3 ... ``` -All images defined in `docker-compose.yml` will be built from the `scidas/slurm.base:19.05.1` base image +All images defined in `docker-compose.yml` are built from the `clustercockpit/slurm.base:24.05.3` base image ## Usage diff --git a/slurm/base/Dockerfile b/slurm/base/Dockerfile index be8aa37..f94f2db 100644 --- a/slurm/base/Dockerfile +++ b/slurm/base/Dockerfile @@ -1,11 +1,9 @@ -FROM rockylinux:8 +FROM almalinux:9 LABEL org.opencontainers.image.authors="jan.eitzinger@fau.de" ENV SLURM_VERSION=24.05.3 -ENV HTTP_PARSER_VERSION=2.8.0 -RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm -RUN ARCH=$(uname -m) && yum install -y https://rpmfind.net/linux/almalinux/8.10/PowerTools/$ARCH/os/Packages/http-parser-devel-2.8.0-9.el8.$ARCH.rpm +RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm RUN groupadd -g 981 munge \ && useradd -m -c "MUNGE Uid 'N' Gid Emporium" -d /var/lib/munge -u 981 -g munge -s /sbin/nologin munge \ @@ -18,11 +16,11 @@ RUN yum install -y munge munge-libs rng-tools \ python3 gcc openssl openssl-devel \ openssh-server openssh-clients dbus-devel \ pam-devel numactl numactl-devel hwloc sudo \ - lua readline-devel ncurses-devel man2html \ - autoconf automake json-c-devel libjwt-devel \ + lua readline-devel ncurses-devel \ + autoconf automake libjwt-devel \ libibmad libibumad rpm-build perl-ExtUtils-MakeMaker.noarch rpm-build make wget -RUN dnf --enablerepo=powertools install -y munge-devel rrdtool-devel lua-devel hwloc-devel mariadb-server mariadb-devel +RUN dnf --enablerepo=crb install -y munge-devel rrdtool-devel lua-devel hwloc-devel mariadb-server mariadb-devel http-parser-devel json-c-devel RUN mkdir -p /usr/local/slurm-tmp \ && cd /usr/local/slurm-tmp \