mirror of
https://github.com/ClusterCockpit/cc-docker.git
synced 2025-03-14 19:05:56 +01:00
Finished testing docker services and updated README
This commit is contained in:
parent
43ca8e35dd
commit
c1b53867c1
28
README.md
28
README.md
@ -12,7 +12,7 @@ It includes the following containers:
|
|||||||
|Slurm Worker|node01|6818|
|
|Slurm Worker|node01|6818|
|
||||||
|MariaDB service|mariadb|3306|
|
|MariaDB service|mariadb|3306|
|
||||||
|InfluxDB serice|influxdb|8086|
|
|InfluxDB serice|influxdb|8086|
|
||||||
|NATS service|nats|4222|
|
|NATS service|nats|4222, 6222, 8222|
|
||||||
|cc-metric-store service|cc-metric-store|8084|
|
|cc-metric-store service|cc-metric-store|8084|
|
||||||
|OpenLDAP|openldap|389, 636|
|
|OpenLDAP|openldap|389, 636|
|
||||||
|
|
||||||
@ -62,23 +62,23 @@ Credentials for the preconfigured demo user are:
|
|||||||
* User: `demo`
|
* User: `demo`
|
||||||
* Password: `demo`
|
* Password: `demo`
|
||||||
|
|
||||||
|
Credentials for the preconfigured LDAP user are:
|
||||||
|
* User: `ldapuser`
|
||||||
|
* Password: `ldapuser`
|
||||||
|
|
||||||
You can also login as regular user using any credential in the LDAP user directory at `./data/ldap/users.ldif`.
|
You can also login as regular user using any credential in the LDAP user directory at `./data/ldap/users.ldif`.
|
||||||
|
|
||||||
## Post-Setup adjustment for using `cc-metric-store`
|
## Preconfigured setup between docker services and ClusterCockpit components
|
||||||
|
|
||||||
When using `influxdb` as a metric database, one must adjust the following files:
|
When you are done cloning the cc-backend repo and once you execute `setupDev.sh` file, it will copy a preconfigured `config.json` from `misc/config.json` and replace the `cc-backend/config.json`, which will be used by cc-backend, once you start the server.
|
||||||
* `cc-backend/var/job-archive/fritz/cluster.json`
|
The preconfigured config.json attaches to:
|
||||||
* `cc-backend/var/job-archive/alex/cluster.json`
|
#### 1. MariaDB docker service on port 3306 (database: ccbackend)
|
||||||
|
#### 2. OpenLDAP docker service on port 389
|
||||||
|
#### 3. cc-metric-store docker service on port 8084
|
||||||
|
|
||||||
In the JSON (cc-backend/config.json), exchange the content of the `metricDataRepository`-Entry (By default configured for `cc-metric-store`) with:
|
cc-metric-store also has a preconfigured `config.json` in `cc-metric-store/config.json` which attaches to NATS docker service on port 4222 and subscribes to topic 'hpc-nats'.
|
||||||
```
|
|
||||||
"metricDataRepository":
|
Basically, all the ClusterCockpit components and the docker services attach to each other like lego pieces.
|
||||||
{
|
|
||||||
"kind": "cc-metric-store",
|
|
||||||
"url": "http://localhost:8082",
|
|
||||||
"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJ1c2VyIjoiYWRtaW4iLCJyb2xlcyI6WyJST0xFX0FETUlOIiwiUk9MRV9BTkFMWVNUIiwiUk9MRV9VU0VSIl19.d-3_3FZTsadPjDEdsWrrQ7nS0edMAR4zjl-eK7rJU3HziNBfI9PDHDIpJVHTNN5E5SlLGLFXctWyKAkwhXL-Dw"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Docker commands to access the services
|
## Docker commands to access the services
|
||||||
|
|
||||||
|
@ -6,6 +6,11 @@ echo "| Starting file required by docker services in data/
|
|||||||
echo "|--------------------------------------------------------------------------------------|"
|
echo "|--------------------------------------------------------------------------------------|"
|
||||||
|
|
||||||
# Download unedited checkpoint files to ./data/cc-metric-store-source/checkpoints
|
# Download unedited checkpoint files to ./data/cc-metric-store-source/checkpoints
|
||||||
|
# After this, migrateTimestamp.pl will run from setupDev.sh. This will update the timestamps
|
||||||
|
# for all the checkpoint files, which then can be read by cc-metric-store.
|
||||||
|
# cc-metric-store reads only data upto certain time, like 48 hours of data.
|
||||||
|
# These checkpoint files have timestamp older than 48 hours and needs to be updated with
|
||||||
|
# migrateTimestamp.pl file, which will be automatically invoked from setupDev.sh.
|
||||||
if [ ! -d data/cc-metric-store-source ]; then
|
if [ ! -d data/cc-metric-store-source ]; then
|
||||||
mkdir -p data/cc-metric-store-source/checkpoints
|
mkdir -p data/cc-metric-store-source/checkpoints
|
||||||
cd data/cc-metric-store-source/checkpoints
|
cd data/cc-metric-store-source/checkpoints
|
||||||
@ -17,6 +22,10 @@ else
|
|||||||
echo "'data/cc-metric-store-source' already exists!"
|
echo "'data/cc-metric-store-source' already exists!"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# A simple configuration file for mariadb docker service.
|
||||||
|
# Required because you can specify only one database per docker service.
|
||||||
|
# This file mentions the database to be created for cc-backend.
|
||||||
|
# This file automatically picked by mariadb after the docker service starts.
|
||||||
if [ ! -d data/mariadb ]; then
|
if [ ! -d data/mariadb ]; then
|
||||||
mkdir -p data/mariadb
|
mkdir -p data/mariadb
|
||||||
cat > data/mariadb/01.databases.sql <<EOF
|
cat > data/mariadb/01.databases.sql <<EOF
|
||||||
@ -26,6 +35,9 @@ else
|
|||||||
echo "'data/mariadb' already exists!"
|
echo "'data/mariadb' already exists!"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# A simple configuration file for openldap docker service.
|
||||||
|
# Creates a simple user 'ldapuser' with password 'ldapuser'.
|
||||||
|
# This file automatically picked by openldap after the docker service starts.
|
||||||
if [ ! -d data/ldap ]; then
|
if [ ! -d data/ldap ]; then
|
||||||
mkdir -p data/ldap
|
mkdir -p data/ldap
|
||||||
cat > data/ldap/add_users.ldif <<EOF
|
cat > data/ldap/add_users.ldif <<EOF
|
||||||
@ -50,6 +62,11 @@ else
|
|||||||
echo "'data/ldap' already exists!"
|
echo "'data/ldap' already exists!"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# A simple configuration file for nats docker service.
|
||||||
|
# Required because we need to execute custom commands after nats docker service starts.
|
||||||
|
# This file automatically executed when the nats docker service starts.
|
||||||
|
# After docker service starts, there is an infinite while loop that publises data for 'fritz' and 'alex' cluster
|
||||||
|
# to subject 'hpc-nats' every 1 minute. Random data is generated only for node level metrics, not hardware level metrics.
|
||||||
if [ ! -d data/nats ]; then
|
if [ ! -d data/nats ]; then
|
||||||
mkdir -p data/nats
|
mkdir -p data/nats
|
||||||
cat > data/nats/docker-entrypoint.sh <<EOF
|
cat > data/nats/docker-entrypoint.sh <<EOF
|
||||||
@ -100,7 +117,7 @@ else
|
|||||||
echo "'data/nats' already exists!"
|
echo "'data/nats' already exists!"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# prepare folders for influxdb2
|
# prepare folders for influxdb3
|
||||||
if [ ! -d data/influxdb ]; then
|
if [ ! -d data/influxdb ]; then
|
||||||
mkdir -p data/influxdb/data
|
mkdir -p data/influxdb/data
|
||||||
mkdir -p data/influxdb/config
|
mkdir -p data/influxdb/config
|
||||||
|
15
setupDev.sh
15
setupDev.sh
@ -16,7 +16,7 @@ echo "| 'sudo' keyword.
|
|||||||
echo "|--------------------------------------------------------------------------------------|"
|
echo "|--------------------------------------------------------------------------------------|"
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
# Check cc-backend, touch job.db if exists
|
# Check cc-backend if exists
|
||||||
if [ ! -d cc-backend ]; then
|
if [ ! -d cc-backend ]; then
|
||||||
echo "'cc-backend' not yet prepared! Please clone cc-backend repository before starting this script."
|
echo "'cc-backend' not yet prepared! Please clone cc-backend repository before starting this script."
|
||||||
echo -n "Stopped."
|
echo -n "Stopped."
|
||||||
@ -44,14 +44,20 @@ else
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Creates data directory if it does not exists.
|
||||||
|
# Contains all the mount points required by all the docker services
|
||||||
|
# and their static files.
|
||||||
if [ ! -d data ]; then
|
if [ ! -d data ]; then
|
||||||
mkdir -m777 data
|
mkdir -m777 data
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Invokes the dataGenerationScript.sh, which then populates the required
|
||||||
|
# static files by the docker services. These static files are required by docker services after startup.
|
||||||
chmod u+x dataGenerationScript.sh
|
chmod u+x dataGenerationScript.sh
|
||||||
./dataGenerationScript.sh
|
./dataGenerationScript.sh
|
||||||
|
|
||||||
# Update timestamps
|
# Update timestamps for all the checkpoints in data/cc-metric-store-source
|
||||||
|
# and dumps new files in data/cc-metric-store.
|
||||||
perl ./migrateTimestamps.pl
|
perl ./migrateTimestamps.pl
|
||||||
|
|
||||||
# Create archive folder for rewritten ccms checkpoints
|
# Create archive folder for rewritten ccms checkpoints
|
||||||
@ -64,13 +70,18 @@ if [ -d data/cc-metric-store-source ]; then
|
|||||||
rm -r data/cc-metric-store-source
|
rm -r data/cc-metric-store-source
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Just in case user forgot manually shutdown the docker services.
|
||||||
docker-compose down
|
docker-compose down
|
||||||
docker-compose down --remove-orphans
|
docker-compose down --remove-orphans
|
||||||
|
|
||||||
|
# This automatically builds the base docker image for slurm.
|
||||||
|
# All the slurm docker service in docker-compose.yml refer to
|
||||||
|
# the base image created from this directory.
|
||||||
cd slurm/base/
|
cd slurm/base/
|
||||||
make
|
make
|
||||||
cd ../..
|
cd ../..
|
||||||
|
|
||||||
|
# Starts all the docker services from docker-compose.yml.
|
||||||
docker-compose build
|
docker-compose build
|
||||||
docker-compose up -d
|
docker-compose up -d
|
||||||
|
|
||||||
|
Loading…
x
Reference in New Issue
Block a user