ClusterCockpit/cc-docker
1
0
Fork 0
mirror of https://github.com/ClusterCockpit/cc-docker.git synced 2026-09-02 04:47:15 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add Keycloak realm import and LDAP role groups

Browse Source 
- Auto-import the clustercockpit realm on Keycloak start
- Rewrite the generated LDAP directory with cc-* role groups and dev users
- Move config.json to the main/nats/auth schema; cc-backend now on :8088
- Add resetDev.sh to tear down containers, volumes and generated data
- Bump cc-metric-store build image to golang 1.26.4
- Ignore all of data/ (generated by dataGenerationScript.sh)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
moebiusband
2026-08-26 21:19:34 +02:00
co-authored by Claude Opus 5
parent a2e7c62115
commit cb86f95b24
12 changed files with 509 additions and 366 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+4 -7
View File
@@ -1,9 +1,6 @@
data/job-archive
data/job-archive/**
data/sqldata
data/cc-metric-store
data/cc-metric-store-source
data/slurm
# everything under data/ is generated by dataGenerationScript.sh
data/
cc-backend/**
.vscode
cc-backend
.vscode
README.md
+114 -10
View File
@@ -70,7 +70,8 @@ If you are using different Linux flavors, you will have to adapt
to start `cc-backend`.
1. By default, you can access `cc-backend` in your browser at
`http://localhost:8080`. You can shut down the cc-backend server by pressing
`http://localhost:8088` (port 8080 is taken by the KeyCloak container). You
can shut down the cc-backend server by pressing
`CTRL-C`, remember to also shut down all containers via `$> docker-compose down`
afterwards.
@@ -83,24 +84,127 @@ Credentials for the preconfigured demo user are:
- User: `demo`
- Password: `demo`
Credentials for the preconfigured LDAP user are:
The LDAP user directory is defined in `./data/ldap/add_users.ldif`. Every
account's password equals its user name:
- User: `ldapuser`
- Password: `ldapuser`
| User | Group | Role in ClusterCockpit |
| ----------- | ------------ | ---------------------- |
| `ldapuser` | – | `user` |
| `ccuser` | – | `user` |
| `ccadmin` | `cc-admin` | `user`, `admin` |
| `ccsupport` | `cc-support` | `user` |
| `ccmanager` | `cc-manager` | `user` |
| `ccapi` | `cc-api` | `user` |
You can also login as regular user using any credential in the LDAP user
directory at `./data/ldap/users.ldif`.
### LDAP role sync
cc-backend can derive elevated roles from LDAP group membership via
`auth.ldap.role-filters` in `config.json`. Each entry maps a role to an LDAP
filter that is evaluated against the *user* entry; the preconfigured setup maps
the `cc-admin` group to the `admin` role:
```json
"role-filters": {
"admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)"
}
```
The `memberOf` attribute is maintained automatically by the `memberof` overlay
of the openldap image, which is configured for `groupOfUniqueNames` /
`uniqueMember` — the group entries in the ldif use those classes accordingly.
LDAP is authoritative for every role listed in `role-filters`: a role is granted
when the filter matches and revoked when it no longer does. Roles that are not
listed are never touched. To exercise the other groups, add them as well:
```json
"role-filters": {
"admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)",
"support": "(memberOf=cn=cc-support,ou=groups,dc=example,dc=com)",
"manager": "(memberOf=cn=cc-manager,ou=groups,dc=example,dc=com)",
"api": "(memberOf=cn=cc-api,ou=groups,dc=example,dc=com)"
}
```
Roles are applied when the account is first synced (`sync-user-on-login`) and
refreshed on every subsequent login (`update-user-on-login`), as well as by the
periodic sync task (`sync-interval`).
> Note: The ldif is only applied when the openldap container initialises its
> database. After changing it, recreate the container and its volumes with
> `docker compose rm -sfv openldap`, or load the changes into the running
> directory manually with `ldapadd`.
### OIDC login via KeyCloak
The KeyCloak container imports the realm `clustercockpit` from
`./keycloak/import/clustercockpit-realm.json` on first start. It defines the
confidential client `cc-backend`, the realm roles `cc-admin`, `cc-support`,
`cc-manager` and `cc-api`, and these accounts (password equals user name):
| User | Realm role | Role in ClusterCockpit |
| ----------- | ------------ | ---------------------- |
| `kcuser` | – | `user` |
| `kcadmin` | `cc-admin` | `admin` |
| `kcsupport` | `cc-support` | `user` |
| `kcmanager` | `cc-manager` | `user` |
| `kcapi` | `cc-api` | `user` |
The KeyCloak admin console is at `http://localhost:8080` (`admin` / `admin`).
The `auth.oidc.role-mapping` section of `config.json` translates realm roles
into ClusterCockpit roles. The preconfigured setup maps `cc-admin` to `admin`:
```json
"oidc": {
"provider": "http://localhost:8080/realms/clustercockpit",
"client-id": "cc-backend",
"client-secret": "cc-backend-dev-secret",
"sync-user-on-login": true,
"update-user-on-login": true,
"role-mapping": {
"cc-admin": "admin"
}
}
```
Unlike the LDAP role filters, `role-mapping` is the *sole* source of roles for
OIDC logins: a role in the token grants a ClusterCockpit role only if it is
listed here, and unmapped roles are ignored — literal role names such as `admin`
would also have to be mapped explicitly. Accounts without a mapped role get
`user`. Add the remaining groups to exercise them:
```json
"role-mapping": {
"cc-admin": "admin",
"cc-support": "support",
"cc-manager": "manager",
"cc-api": "api"
}
```
Roles are read from the `realm_access.roles` claim of the **ID token**. The
built-in KeyCloak `roles` client scope only adds that claim to the *access*
token, so the imported client carries its own `realm roles in id token`
protocol mapper with `id.token.claim` enabled. Keep that mapper when editing
the realm, otherwise every OIDC login falls back to the plain `user` role.
> Note: `cc-backend` resolves the OIDC provider at startup and aborts if it is
> unreachable, so the KeyCloak container has to be up before starting the
> server. The realm is only imported while the KeyCloak database is empty —
> to re-import after editing the realm file, recreate KeyCloak *and* its
> database with `docker compose rm -sf keycloak postgres`.
## Preconfigured setup between docker services and ClusterCockpit components
When you are done cloning the cc-backend repo and once you execute `setupDev.sh` file, it will copy a preconfigured `config.json` from `misc/config.json` and replace the `cc-backend/config.json`, which will be used by cc-backend, once you start the server.
The preconfigured config.json attaches to:
#### 1. OpenLDAP docker service on port 389
### 1. OpenLDAP docker service on port 389
#### 2. cc-metric-store docker service on port 8084
### 2. cc-metric-store docker service on port 8084
#### 3. cc-slurm-adapter is running on slurmctld docker service
### 3. cc-slurm-adapter is running on slurmctld docker service
cc-metric-store also has a preconfigured `config.json` in
`cc-metric-store/config.json` which attaches to NATS docker service on port 4222
@@ -194,7 +298,7 @@ custom CURL commands.
## Known Issues
- `docker-compose` installed on Ubuntu (18.04, 20.04) via `apt-get` can not correctly parse `docker-compose.yml` due to version differences. Install latest version of `docker-compose` from <https://docs.docker.com/compose/install/> instead.
- You need to ensure that no other web server is running on ports 8080 (cc-backend), 8084 (cc-metric-store), 4222 and 8222 (Nats). If one or more ports are already in use, you have to adapt the related config accordingly.
- You need to ensure that no other web server is running on ports 8088 (cc-backend), 8080 (KeyCloak), 8084 (cc-metric-store), 4222 and 8222 (Nats). If one or more ports are already in use, you have to adapt the related config accordingly.
- Existing VPN connections sometimes cause problems with docker. If `docker-compose` does not start up correctly, try disabling any active VPN connection. Refer to <https://stackoverflow.com/questions/45692255/how-make-openvpn-work-with-docker> for further information.
## Docker services and restarting the services
cc-metric-store/Dockerfile
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.22.4
FROM golang:1.26.4
RUN apt-get update
RUN apt-get -y install git
data/ldap/bootstrap.ldif
-239
View File
@@ -1,239 +0,0 @@
# ClusterCockpit Bootstrap LDAP Directory
# =========================================
# Domain: dc=example,dc=com (LDAP_DOMAIN=example.com in docker-compose.yml)
# Admin DN: cn=admin,dc=example,dc=com (set via LDAP_ADMIN_PASSWORD env)
#
# All test user passwords: "password"
# {SHA} hash verification: slappasswd -h {SHA} -s password
#
# Suggested cc-backend ldap config (config.json):
# "url": "ldap://ldap:389"
# "user-base": "ou=people,dc=example,dc=com"
# "search-dn": "uid=ccbinduser,ou=people,dc=example,dc=com"
# "user-bind": "uid={username},ou=people,dc=example,dc=com"
# "user-filter": "(&(objectclass=posixAccount)(!(uid=ccbinduser)))"
# "username-attr": "gecos"
# "uid-attr": "uid"
# "sync-password": "password"
#
# ClusterCockpit roles (from cc-lib/schema/user.go):
# anonymous < api < user < manager < support < admin
# =========================================
# ---------------------------------------------------------------------------
# Organizational Units
# ---------------------------------------------------------------------------
dn: ou=people,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: people
description: HPC user accounts
dn: ou=groups,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: groups
description: HPC project groups and ClusterCockpit role groups
# ---------------------------------------------------------------------------
# Service account used by cc-backend for LDAP search binding
# ---------------------------------------------------------------------------
dn: uid=ccbinduser,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC Bind User
sn: BindUser
uid: ccbinduser
uidNumber: 500
gidNumber: 500
homeDirectory: /home/ccbinduser
description: Service account for cc-backend LDAP search
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# ---------------------------------------------------------------------------
# Test users
# Role membership is tracked via cc-role-* groups below.
# ---------------------------------------------------------------------------
# admin01 — ClusterCockpit admin
dn: uid=admin01,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Admin User
sn: User
uid: admin01
uidNumber: 1001
gidNumber: 1001
homeDirectory: /home/admin01
gecos: Admin User
mail: admin01@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# support01 — ClusterCockpit support staff
dn: uid=support01,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Support User
sn: User
uid: support01
uidNumber: 1002
gidNumber: 1001
homeDirectory: /home/support01
gecos: Support User
mail: support01@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# manager01 — ClusterCockpit project manager
dn: uid=manager01,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Manager User
sn: User
uid: manager01
uidNumber: 1003
gidNumber: 1001
homeDirectory: /home/manager01
gecos: Manager User
mail: manager01@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# user01 — regular HPC user
dn: uid=user01,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Regular User 01
sn: User
uid: user01
uidNumber: 1010
gidNumber: 1001
homeDirectory: /home/user01
gecos: Regular User 01
mail: user01@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# user02 — regular HPC user (also member of a project group)
dn: uid=user02,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Regular User 02
sn: User
uid: user02
uidNumber: 1011
gidNumber: 1001
homeDirectory: /home/user02
gecos: Regular User 02
mail: user02@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# user03 — regular HPC user (also member of a project group)
dn: uid=user03,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: Regular User 03
sn: User
uid: user03
uidNumber: 1012
gidNumber: 1001
homeDirectory: /home/user03
gecos: Regular User 03
mail: user03@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# apiuser01 — programmatic/service API access
dn: uid=apiuser01,ou=people,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: API User 01
sn: User
uid: apiuser01
uidNumber: 1020
gidNumber: 1001
homeDirectory: /home/apiuser01
gecos: API User 01
mail: apiuser01@example.com
userPassword: {SHA}W6ph5Mm5Pz8GgiULbPgzG37mj9g=
# ---------------------------------------------------------------------------
# ClusterCockpit role groups
# These map to cc-lib Role constants: admin, support, manager, user, api
# cc-backend can use these for group-based user filtering or future role sync.
# Example user-filter to restrict login to group members:
# (&(objectclass=posixAccount)(memberOf=cn=cc-users,ou=groups,dc=example,dc=com))
# Note: memberOf requires the memberof overlay; use memberUid for posixGroup.
# ---------------------------------------------------------------------------
dn: cn=cc-admins,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: cc-admins
gidNumber: 2000
description: ClusterCockpit administrators (role: admin)
memberUid: admin01
dn: cn=cc-support,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: cc-support
gidNumber: 2001
description: ClusterCockpit support staff (role: support)
memberUid: support01
dn: cn=cc-managers,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: cc-managers
gidNumber: 2002
description: ClusterCockpit project managers (role: manager)
memberUid: manager01
dn: cn=cc-users,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: cc-users
gidNumber: 2003
description: ClusterCockpit regular users (role: user)
memberUid: user01
memberUid: user02
memberUid: user03
dn: cn=cc-api,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: cc-api
gidNumber: 2004
description: ClusterCockpit API/service accounts (role: api)
memberUid: apiuser01
# ---------------------------------------------------------------------------
# HPC project groups (for testing manager project-scoping)
# A manager assigned to project hpc_proj_alpha can view all jobs in that project.
# ---------------------------------------------------------------------------
dn: cn=hpc_proj_alpha,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: hpc_proj_alpha
gidNumber: 3001
description: HPC project alpha
memberUid: manager01
memberUid: user01
memberUid: user02
dn: cn=hpc_proj_beta,ou=groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: hpc_proj_beta
gidNumber: 3002
description: HPC project beta
memberUid: manager01
memberUid: user03
dataGenerationScript.sh
+129 -7
View File
@@ -37,16 +37,44 @@ else
echo "'data/mariadb' already exists!"
fi
# A simple configuration file for openldap docker service.
# Creates a simple user 'ldapuser' with password 'ldapuser'.
# This file automatically picked by openldap after the docker service starts.
# Configuration file for the openldap docker service.
# Creates the dev users (password == uid) and the cc-* role groups used by
# auth.ldap.role-filters in config.json.
# This file is automatically picked up by openldap on first start.
if [ ! -d data/ldap ]; then
mkdir -p data/ldap
cat >data/ldap/add_users.ldif <<EOF
cat >data/ldap/add_users.ldif <<'LDIF'
# ClusterCockpit dev LDAP directory
# =================================
# Applied by the osixia/openldap container on first init via
# /container/service/slapd/assets/config/bootstrap/ldif/custom.
#
# Every account's password equals its uid (e.g. ccadmin / ccadmin).
# Hashes are {SSHA}; regenerate with `slappasswd -h {SSHA} -s <password>`.
#
# Role sync (auth.ldap.role-filters in config.json) matches on the memberOf
# attribute of the *user* entry. The memberof overlay of this image maintains
# memberOf for groupOfUniqueNames/uniqueMember, so the group entries below must
# come after the user entries they reference.
# ---------------------------------------------------------------------------
# Organizational units
# ---------------------------------------------------------------------------
dn: ou=users,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: users
dn: ou=groups,dc=example,dc=com
objectClass: organizationalUnit
objectClass: top
ou: groups
# ---------------------------------------------------------------------------
# Users
# ---------------------------------------------------------------------------
dn: uid=ldapuser,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
@@ -55,10 +83,104 @@ cn: Ldap User
sn: User
uid: ldapuser
uidNumber: 1
gidNumber: 1
gidNumber: 100
homeDirectory: /home/ldapuser
userPassword: {SSHA}sQRqFQtuiupej7J/rbrQrTwYEHDduV+N
EOF
userPassword: {SSHA}0OhsiEb+IXanz71bd7jyFhvy0xhPJ34e
dn: uid=ccadmin,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC Admin
sn: Admin
uid: ccadmin
uidNumber: 2000
gidNumber: 100
homeDirectory: /home/ccadmin
userPassword: {SSHA}HfIVI9GeNot9ea5bSXgKNcF6kP0MUyfv
dn: uid=ccsupport,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC Support
sn: Support
uid: ccsupport
uidNumber: 2001
gidNumber: 100
homeDirectory: /home/ccsupport
userPassword: {SSHA}7zCZNDYQZPZl8jcBSkU+UnmxCR8lD9sm
dn: uid=ccmanager,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC Manager
sn: Manager
uid: ccmanager
uidNumber: 2002
gidNumber: 100
homeDirectory: /home/ccmanager
userPassword: {SSHA}DKcUb2h8DWbDtTjXtIvwDi4Bf29vzhKK
dn: uid=ccapi,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC Api
sn: Api
uid: ccapi
uidNumber: 2003
gidNumber: 100
homeDirectory: /home/ccapi
userPassword: {SSHA}dhcxIxs883hX2CTpkQp7cNgc0+puJ23z
dn: uid=ccuser,ou=users,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: top
cn: CC User
sn: User
uid: ccuser
uidNumber: 2004
gidNumber: 100
homeDirectory: /home/ccuser
userPassword: {SSHA}UKYfiDmhT/H01R20rtYYOBj7t/w7yLyW
# ---------------------------------------------------------------------------
# Role groups
# The memberof overlay of this image is configured for groupOfUniqueNames /
# uniqueMember, so these classes are required for memberOf to be maintained.
# ---------------------------------------------------------------------------
dn: cn=cc-admin,ou=groups,dc=example,dc=com
objectClass: groupOfUniqueNames
objectClass: top
cn: cc-admin
description: Grants the ClusterCockpit admin role
uniqueMember: uid=ccadmin,ou=users,dc=example,dc=com
dn: cn=cc-support,ou=groups,dc=example,dc=com
objectClass: groupOfUniqueNames
objectClass: top
cn: cc-support
description: Grants the ClusterCockpit support role
uniqueMember: uid=ccsupport,ou=users,dc=example,dc=com
dn: cn=cc-manager,ou=groups,dc=example,dc=com
objectClass: groupOfUniqueNames
objectClass: top
cn: cc-manager
description: Grants the ClusterCockpit manager role
uniqueMember: uid=ccmanager,ou=users,dc=example,dc=com
dn: cn=cc-api,ou=groups,dc=example,dc=com
objectClass: groupOfUniqueNames
objectClass: top
cn: cc-api
description: Grants the ClusterCockpit api role
uniqueMember: uid=ccapi,ou=users,dc=example,dc=com
LDIF
else
echo "'data/ldap' already exists!"
docker-compose.yml
+5 -1
View File
@@ -57,7 +57,11 @@ services:
ports:
- "0.0.0.0:8080:8080"
restart: always
command: --verbose start --optimized
depends_on:
- postgres
volumes:
- ./keycloak/import:/opt/keycloak/data/import:ro
command: --verbose start --optimized --import-realm
mariadb:
container_name: mariadb
keycloak/import/clustercockpit-realm.json
+111
View File
@@ -0,0 +1,111 @@
{
"realm": "clustercockpit",
"displayName": "ClusterCockpit",
"enabled": true,
"sslRequired": "none",
"registrationAllowed": false,
"loginWithEmailAllowed": true,
"roles": {
"realm": [
{ "name": "cc-admin", "description": "Grants the ClusterCockpit admin role" },
{ "name": "cc-support", "description": "Grants the ClusterCockpit support role" },
{ "name": "cc-manager", "description": "Grants the ClusterCockpit manager role" },
{ "name": "cc-api", "description": "Grants the ClusterCockpit api role" }
]
},
"clients": [
{
"clientId": "cc-backend",
"name": "ClusterCockpit Backend",
"description": "Confidential OIDC client used by cc-backend",
"enabled": true,
"protocol": "openid-connect",
"publicClient": false,
"secret": "cc-backend-dev-secret",
"standardFlowEnabled": true,
"directAccessGrantsEnabled": true,
"implicitFlowEnabled": false,
"serviceAccountsEnabled": false,
"redirectUris": [
"http://localhost:8088/oidc-callback",
"http://127.0.0.1:8088/oidc-callback"
],
"webOrigins": [
"http://localhost:8088",
"http://127.0.0.1:8088"
],
"attributes": {
"pkce.code.challenge.method": "S256",
"post.logout.redirect.uris": "http://localhost:8088/*"
},
"protocolMappers": [
{
"name": "realm roles in id token",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-realm-role-mapper",
"consentRequired": false,
"config": {
"multivalued": "true",
"claim.name": "realm_access.roles",
"jsonType.label": "String",
"id.token.claim": "true",
"access.token.claim": "true",
"userinfo.token.claim": "false"
}
}
]
}
],
"users": [
{
"username": "kcadmin",
"enabled": true,
"emailVerified": true,
"firstName": "KC",
"lastName": "Admin",
"email": "kcadmin@example.com",
"credentials": [{ "type": "password", "value": "kcadmin", "temporary": false }],
"realmRoles": ["default-roles-clustercockpit", "cc-admin"]
},
{
"username": "kcsupport",
"enabled": true,
"emailVerified": true,
"firstName": "KC",
"lastName": "Support",
"email": "kcsupport@example.com",
"credentials": [{ "type": "password", "value": "kcsupport", "temporary": false }],
"realmRoles": ["default-roles-clustercockpit", "cc-support"]
},
{
"username": "kcmanager",
"enabled": true,
"emailVerified": true,
"firstName": "KC",
"lastName": "Manager",
"email": "kcmanager@example.com",
"credentials": [{ "type": "password", "value": "kcmanager", "temporary": false }],
"realmRoles": ["default-roles-clustercockpit", "cc-manager"]
},
{
"username": "kcapi",
"enabled": true,
"emailVerified": true,
"firstName": "KC",
"lastName": "Api",
"email": "kcapi@example.com",
"credentials": [{ "type": "password", "value": "kcapi", "temporary": false }],
"realmRoles": ["default-roles-clustercockpit", "cc-api"]
},
{
"username": "kcuser",
"enabled": true,
"emailVerified": true,
"firstName": "KC",
"lastName": "User",
"email": "kcuser@example.com",
"credentials": [{ "type": "password", "value": "kcuser", "temporary": false }],
"realmRoles": ["default-roles-clustercockpit"]
}
]
}
misc/config.json
+65 -59
View File
@@ -1,25 +1,10 @@
{
"addr": "127.0.0.1:8080",
"main": {
"addr": "0.0.0.0:8088",
"short-running-jobs-duration": 300,
"archive": {
"kind": "file",
"path": "./var/job-archive"
},
"jwts": {
"max-age": "2000h"
},
"apiAllowedIPs": [
"*"
],
"ldap": {
"url": "ldap://0.0.0.0",
"user_base": "ou=users,dc=example,dc=com",
"search_dn": "cn=admin,dc=example,dc=com",
"user_bind": "uid={username},ou=users,dc=example,dc=com",
"user_filter": "(&(objectclass=posixAccount))",
"syncUserOnLogin": true
},
"enable-resampling": {
"emission-constant": 317,
"resampling": {
"minimum-points": 600,
"trigger": 30,
"resolutions": [
600,
@@ -28,51 +13,72 @@
60
]
},
"emission-constant": 317,
"clusters": [
"api-subjects": {
"subject-job-event": "cc.job.event",
"subject-node-state": "cc.node.state"
}
},
"nats": {
"address": "nats://0.0.0.0:4222",
"username": "root",
"password": "root"
},
"auth": {
"jwts": {
"max-age": "2000h",
"public-key": "kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0=",
"private-key": "dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="
},
"oidc": {
"provider": "http://localhost:8080/realms/clustercockpit",
"client-id": "cc-backend",
"client-secret": "cc-backend-dev-secret",
"sync-user-on-login": true,
"update-user-on-login": true,
"role-mapping": {
"cc-admin": "admin"
}
},
"ldap": {
"url": "ldap://ldap:389",
"user-base": "ou=users,dc=example,dc=com",
"search-dn": "cn=admin,dc=example,dc=com",
"user-bind": "uid={username},ou=users,dc=example,dc=com",
"user-filter": "(objectclass=posixAccount)",
"username-attr": "cn",
"uid-attr": "uid",
"sync-password": "mashup",
"sync-interval": "2h",
"sync-user-on-login": true,
"update-user-on-login": true,
"role-filters": {
"admin": "(memberOf=cn=cc-admin,ou=groups,dc=example,dc=com)"
}
}
},
"cron": {
"commit-job-worker": "1m",
"duration-worker": "5m",
"footprint-worker": "10m"
},
"archive": {
"kind": "file",
"path": "./var/job-archive"
},
"metric-store-external": [
{
"name": "fritz",
"metricDataRepository": {
"kind": "cc-metric-store",
"scope": "fritz",
"url": "http://0.0.0.0:8084",
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"
},
"filterRanges": {
"numNodes": {
"from": 1,
"to": 64
},
"duration": {
"from": 0,
"to": 86400
},
"startTime": {
"from": "2022-01-01T00:00:00Z",
"to": null
}
}
},
{
"name": "alex",
"metricDataRepository": {
"kind": "cc-metric-store",
"scope": "alex",
"url": "http://0.0.0.0:8084",
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"
},
"filterRanges": {
"numNodes": {
"from": 1,
"to": 64
},
"duration": {
"from": 0,
"to": 86400
},
"startTime": {
"from": "2022-01-01T00:00:00Z",
"to": null
}
],
"metric-store": {
"retention-in-memory": "24h",
"memory-cap": 100
}
}
}
]
}
resetDev.sh
Executable
+58
View File
@@ -0,0 +1,58 @@
#!/bin/bash
set -euo pipefail
echo ""
echo "|--------------------------------------------------------------------------------------|"
echo "| cc-docker dev environment reset script |"
echo "| This will stop all containers, remove volumes, and delete all generated data. |"
echo "| After completion, re-run ./setupDev.sh to reinitialize. |"
echo "|--------------------------------------------------------------------------------------|"
echo ""
read -p "This will delete all generated data and cannot be undone. Continue? [y/N] " -r
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 0
fi
DOCKER_COMPOSE=""
if docker-compose --version &>/dev/null 2>&1; then
DOCKER_COMPOSE="docker-compose"
fi
if docker compose version &>/dev/null 2>&1; then
DOCKER_COMPOSE="docker compose"
fi
if [[ -z "${DOCKER_COMPOSE}" ]]; then
echo "docker-compose not found!"
exit 1
fi
echo "Stopping containers and removing volumes..."
$DOCKER_COMPOSE down --volumes --remove-orphans
echo "Removing data directory..."
if [ -d data ]; then
rm -rf data
fi
echo "Removing cc-backend generated files..."
if [ -d cc-backend/var ]; then
rm -rf cc-backend/var
fi
if [ -f cc-backend/.env ]; then
rm cc-backend/.env
fi
if [ -f cc-backend/config.json ]; then
rm cc-backend/config.json
fi
echo ""
echo "|--------------------------------------------------------------------------------------|"
echo "| Reset complete. |"
echo "| Run ./setupDev.sh to reinitialize the dev environment. |"
echo "| To also remove built Docker images, run: docker-compose down --rmi all |"
echo "|--------------------------------------------------------------------------------------|"
echo ""
setupDev.sh
-5
View File
@@ -65,12 +65,8 @@ if [ ! -d var ]; then
wget https://hpc-mover.rrze.uni-erlangen.de/HPC-Data/0x7b58aefb/eig7ahyo6fo2bais0ephuf2aitohv1ai/job-archive-demo.tar
tar xf job-archive-demo.tar
rm ./job-archive-demo.tar
cp ./configs/env-template.txt .env
cp ../misc/config.json config.json
sed -i 's/"addr": *"127\.0\.0\.1:8080"/"addr": "0.0.0.0:8080"/' config.json
make
./cc-backend -migrate-db
@@ -113,7 +109,6 @@ cd ../..
$DOCKER_COMPOSE build
$DOCKER_COMPOSE up -d
echo ""
echo "|--------------------------------------------------------------------------------------|"
echo "| Check logs for each slurm service by using these commands: |"
slurm/README.md
+6 -19
View File
@@ -4,15 +4,11 @@
Use [Docker](https://www.docker.com/) to explore the various components of [Slurm](https://www.schedmd.com/index.php)
This work represents a small exploratory Slurm cluster using CentOS 7 based Docker images. The intent was to learn the basics of Slurm prior to extending the concept to a more distributed environment.
This work represents a small exploratory Slurm cluster using AlmaLinux 9 based Docker images.
Images include:
- [Slurm 19.05.1](https://slurm.schedmd.com) - installed from [rpm packages](packages)
- [OpenMPI 3.0.1](https://www.open-mpi.org/doc/current/) - installed from [rpm packages](packages)
- [Lmod 7.7](http://lmod.readthedocs.io/en/latest/index.html) - installed from [distribution files](https://sourceforge.net/projects/lmod/files/)
- [Lmod module packages for CentOS 7](https://github.com/scidas/lmod-modules-centos) - Organized for Slurm-in-Docker use
- [Using Lmod with Slurm-in-Docker](using-lmod-with-slurm-in-docker.md) documentation
- [Slurm 24.05.3](https://slurm.schedmd.com) - built from source RPMs inside the container
## Contents
@@ -63,33 +59,24 @@ TODO: Have software check validity of custom configuration files.
## Build
Build the slurm RPM files by following the instructions in the [packages](packages) directory.
**Create the base Slurm image**:
Copy the `packages/centos-7/rpms` directory to the `base` directory
```
cd base/
cp -r ../packages/centos-7/rpms .
make
```
Build the base image
```
docker build -t scidas/slurm.base:19.05.1 .
```
This builds `clustercockpit/slurm.base:24.05.3` from AlmaLinux 9, compiling Slurm RPMs from source with JWT and slurmrestd support.
Verify image build
```console
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
scidas/slurm.base 19.05.1 1600621cb483 Less than a second ago 819MB
...
clustercockpit/slurm.base 24.05.3 ...
```
All images defined in `docker-compose.yml` will be built from the `scidas/slurm.base:19.05.1` base image
All images defined in `docker-compose.yml` are built from the `clustercockpit/slurm.base:24.05.3` base image
## Usage
slurm/base/Dockerfile
+5 -7
View File
@@ -1,11 +1,9 @@
FROM rockylinux:8
FROM almalinux:9
LABEL org.opencontainers.image.authors="jan.eitzinger@fau.de"
ENV SLURM_VERSION=24.05.3
ENV HTTP_PARSER_VERSION=2.8.0
RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
RUN ARCH=$(uname -m) && yum install -y https://rpmfind.net/linux/almalinux/8.10/PowerTools/$ARCH/os/Packages/http-parser-devel-2.8.0-9.el8.$ARCH.rpm
RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
RUN groupadd -g 981 munge \
&& useradd -m -c "MUNGE Uid 'N' Gid Emporium" -d /var/lib/munge -u 981 -g munge -s /sbin/nologin munge \
@@ -18,11 +16,11 @@ RUN yum install -y munge munge-libs rng-tools \
python3 gcc openssl openssl-devel \
openssh-server openssh-clients dbus-devel \
pam-devel numactl numactl-devel hwloc sudo \
lua readline-devel ncurses-devel man2html \
autoconf automake json-c-devel libjwt-devel \
lua readline-devel ncurses-devel \
autoconf automake libjwt-devel \
libibmad libibumad rpm-build perl-ExtUtils-MakeMaker.noarch rpm-build make wget
RUN dnf --enablerepo=powertools install -y munge-devel rrdtool-devel lua-devel hwloc-devel mariadb-server mariadb-devel
RUN dnf --enablerepo=crb install -y munge-devel rrdtool-devel lua-devel hwloc-devel mariadb-server mariadb-devel http-parser-devel json-c-devel
RUN mkdir -p /usr/local/slurm-tmp \
&& cd /usr/local/slurm-tmp \