Skip to main content

Deploying Salt.Box Server

Prepare​

  1. Clone the salt-box-compose project into your working directory:
git clone https://dev.saltbox.pro/salt.box/salt-box-compose.git

  1. Change the current working directory:

    cd ./salt-box-compose/

  2. Initially some secrets need to be created in the ./secrets/ subdirectory. This may be done with helper script:

    python3 ./bin/make_secrets.py

  3. Make a copy of example.env file named .env in the project root. Tune it before run building.

Run​

  1. While in the ./salt-box-compose/ directory, start the build process:

    sudo docker compose up --build -d
    caution

    Check there are no warnings on non-set variables to avoid confusing errors.

  2. To update outer images:

    sudo docker compose pull

    then restart e.g. with:

    sudo docker compose down && sudo docker compose up -d

    It is possible to pull images on start with an additional flag --pull=always:

    sudo docker compose up --build --pull=always

    For production use following commands are recommended:

    sudo sh -c "echo 'vm.overcommit_memory=1' > /etc/sysctl.d/salt-box.conf"
    sudo sysctl -p /etc/sysctl.d/salt-box.conf

    vm.overcommit_memory=1 is a Redis requirement.

Docker objects that are being created during installation​

Once the build process has completed successfully, a number of Docker objects are created, as shown in the table below.

Docker ObjectStatusDescription
make-redis-salt-certs—
proxy—
Network salt-box_default—Internal network for communication between Docker containers
VOLUMES:
f85f6baa5742b4b1bf7b3b7ca86929361c
529aee6e06bff9b1a5208d76469345		—
salt-box_salt_box_core_var—
(the owning container: salt-box-salt-box-core-1)
salt-box_keycloak-db_data—KeyCloak service data
(the owning container: salt-box-keycloak-1)
salt-box_salt_master_pki—salt-master service keystore
(the owning container: salt-box-salt-master-1)
salt-box_redis_taskiq_data—taskiq service data
(the owning container: salt-box-salt-box-core-taskiq-worker-1)
salt-box_mongo_config—MongoDB configuration data
(the owning container: salt-box-mongo-1)
salt-box_mongo_data—MongoDB data
(the owning container: salt-box-mongo-1)
salt-box_redis_salt_data—Redis server data
(the owning container: salt-box-redis-salt-1)
salt-box_redis_salt_certs—Redis server certificate store
(the owning container: salt-box-redis-salt-1)
CONTAINERS:
salt-box-mongo-1UpThe container runs the MongoDB DBMS.
It is used by the Salt.Box backend.
The DB stores entities such as clients, collections, etc.
salt-box-sshfs-1UpThe SSH server is running in this container.
It is used to mount the sshfs file systems on minions
salt-box-opa-1UpThe Open Policy Agent is running in this container.
OPA is used to create access policies for different parts of the web application
salt-box-keycloak-db-1UpThe KeyCloak service is running in this container.
It provides centralized authentication in the Salt.Box web interface
salt-box-salt-box-frontend-1UpSalt.Box user interface (frontend)
salt-box-make-redis-salt-certs-1ExitedA module that issues certificates for Redis Certificate Based Authentication
salt-box-salt-box-core-tasks-watcher-1UpA module waiting for tasks to appear on the SaltStack exchange bus
salt-box-salt-box-core-taskiq-scheduler-1UpModule responsible for working with a distributed asynchronous task queue
salt-box-salt-box-core-taskiq-worker-1UpThe tasiq worker process is running in this container.
It is responsible for receiving, sending, and executing tasks from the queue
salt-box-rabbitmq-1UpThe RabbitMQ service is running in this container.
RabbitMQ is a message broker.
It is responsible for collecting data from multiple sources and sending it to services of the Salt.Box system for further processing
salt-box-redis-taskiq-1Up
salt-box-salt-box-core-faststream-redis-1Up
salt-box-redis-salt-1UpThe Redis server is running in this container.
Redis server is a resident NoSQL database management system that works with key-value data structures.
It is used to store the results of SaltStack-related processes
salt-box-keycloak-1UpIdAM subsystem of Salt.Box based on KeyCloak
salt-box-salt-box-core-1UpMain server component of Salt.Box (backend)
salt-box-salt-master-1UpThe salt-master daemon, used to control minions
salt-box-keycloak-init-1ExitedHelper scripts that initialize KeyCloak
salt-box-proxy-1UpNginx reverse proxy.
Redirects client requests to internal services in containers

Here the statuses mean:
Up — The container has been successfully launched, and continues to run in the background
Exited — The container was successfully launched, and then stopped normally until demand

Images​

Below is an example of the output of the docker image ls command.

REPOSITORY                                                     TAG            IMAGE ID       CREATED        SIZE
registry.saltbox.pro/salt.box/salt-box-bridge/salt-master      master         6e6a65ae6d5c   14 hours ago   305MB
registry.saltbox.pro/salt.box/salt-box-core                    master         dc990973edb6   15 hours ago   486MB
registry.saltbox.pro/salt.box/salt-box-frontend                master         cb046f5104d3   22 hours ago   215MB
salt-box-proxy                                                 latest         0a7a69e50215   9 days ago     208MB
registry.saltbox.pro/salt.box/salt-box-compose/keycloak-init   master         c3ad347807b2   11 days ago    819MB
registry.saltbox.pro/salt.box/salt-box-compose/keycloak        master         cce07c6262d2   11 days ago    808MB
registry.saltbox.pro/salt.box/salt-box-compose/sshfs           master         6d4bced308df   11 days ago    218MB
registry.saltbox.pro/salt.box/salt-box-compose/opa             master         9f4e528e02fb   11 days ago    82.6MB
registry.saltbox.pro/salt.box/salt-box-compose/redis           master         2cb6940da7ee   11 days ago    60.7MB
mongo                                                          7              8c30f14b362a   2 weeks ago    1.1GB
postgres                                                       16-alpine      3b057e1c2c6d   6 weeks ago    394MB
salt-box-make-redis-salt-certs                                 latest         f82ed3daf965   6 weeks ago    109MB
rabbitmq                                                       3-management   49895caee5d3   6 months ago   389MB

Volumes​

Below is an example of the output of the docker volume ls command.

DRIVER    VOLUME NAME
local     f85f6baa5742b4b1bf7b3b7ca86929361c529aee6e06bff9b1a5208d76469345
local     salt-box_keycloak-db_data
local     salt-box_mongo_config
local     salt-box_mongo_data
local     salt-box_redis_salt_certs
local     salt-box_redis_salt_data
local     salt-box_redis_taskiq_data
local     salt-box_salt_box_core_var
local     salt-box_salt_master_pki

Containers​

Below is an example of the output of the docker compose ls -a command.

NAME                STATUS                   CONFIG FILES
salt-box            exited(2), running(16)   root/salt-box-compose/compose.yaml

Below is an example of the output of the docker container ls -a command.

CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED          STATUS                      PORTS                                                                  NAMES
88dc531cc401   salt-box-proxy                                                        "/docker/entrypoint.…"   22 minutes ago   Up 21 minutes               0.0.0.0:80->80/tcp                                                     salt-box-proxy-1
9027787dcf67   registry.saltbox.pro/salt.box/salt-box-core:master                    "entrypoint.sh salt-…"   22 minutes ago   Up 22 minutes               8000/tcp                                                               salt-box-salt-box-core-faststream-redis-1
372acea8af4a   registry.saltbox.pro/salt.box/salt-box-core:master                    "/usr/local/bin/uvic…"   22 minutes ago   Up 21 minutes               8000/tcp                                                               salt-box-salt-box-core-1
04ccd3ce9c7c   registry.saltbox.pro/salt.box/salt-box-bridge/salt-master:master      "/usr/local/bin/entr…"   22 minutes ago   Up 22 minutes               0.0.0.0:4505-4506->4505-4506/tcp, 8000/tcp                             salt-box-salt-master-1
a864aec0556a   registry.saltbox.pro/salt.box/salt-box-core:master                    "entrypoint.sh salt-…"   22 minutes ago   Up 22 minutes               8000/tcp                                                               salt-box-salt-box-core-tasks-watcher-1
8ed8856d31c1   registry.saltbox.pro/salt.box/salt-box-compose/keycloak-init:master   "/usr/local/bin/crea…"   22 minutes ago   Exited (0) 21 minutes ago                                                                          salt-box-keycloak-init-1
332bc7787b30   registry.saltbox.pro/salt.box/salt-box-core:master                    "/usr/local/bin/task…"   22 minutes ago   Up 21 minutes               8000/tcp                                                               salt-box-salt-box-core-taskiq-scheduler-1
140c425ca829   registry.saltbox.pro/salt.box/salt-box-core:master                    "/usr/local/bin/task…"   22 minutes ago   Up 21 minutes               8000/tcp                                                               salt-box-salt-box-core-taskiq-worker-1
f8b895c79ec2   registry.saltbox.pro/salt.box/salt-box-compose/redis:master           "entrypoint.sh '--sa…"   22 minutes ago   Up 22 minutes (healthy)     127.0.0.1:6379->6379/tcp                                               salt-box-redis-salt-1
05bb59635b38   registry.saltbox.pro/salt.box/salt-box-compose/keycloak:master        "/usr/local/bin/star…"   22 minutes ago   Up 21 minutes (healthy)     8080/tcp                                                               salt-box-keycloak-1
b68811fa1e30   postgres:16-alpine                                                    "docker-entrypoint.s…"   22 minutes ago   Up 22 minutes (healthy)     5432/tcp                                                               salt-box-keycloak-db-1
e7091bac8749   rabbitmq:3-management                                                 "docker-entrypoint.s…"   22 minutes ago   Up 22 minutes (healthy)     4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   salt-box-rabbitmq-1
6673e7cb5a06   mongo:7                                                               "docker-entrypoint.s…"   22 minutes ago   Up 22 minutes               127.0.0.1:27017->27017/tcp                                             salt-box-mongo-1
8cd31f8c27ad   salt-box-make-redis-salt-certs                                        "entrypoint.sh /root…"   22 minutes ago   Exited (0) 22 minutes ago                                                                          salt-box-make-redis-salt-certs-1
b1ba6ea04662   registry.saltbox.pro/salt.box/salt-box-compose/redis:master           "entrypoint.sh '--sa…"   22 minutes ago   Up 22 minutes               6379/tcp                                                               salt-box-redis-taskiq-1
77199f687853   registry.saltbox.pro/salt.box/salt-box-frontend:master                "/docker/entrypoint.…"   22 minutes ago   Up 22 minutes               80/tcp                                                                 salt-box-salt-box-frontend-1
c727e1df0c0b   registry.saltbox.pro/salt.box/salt-box-compose/sshfs:master           "/usr/local/bin/sshd…"   22 minutes ago   Up 22 minutes               0.0.0.0:1022->1022/tcp                                                 salt-box-sshfs-1
53f1daabbebd   registry.saltbox.pro/salt.box/salt-box-compose/opa:master             "/opa run --server -…"   22 minutes ago   Up 22 minutes               0.0.0.0:8181->8181/tcp                                                 salt-box-opa-1

Autotests​

To run test suites enable compose-autotests.yaml in the local .env file.
Then execute:

sudo docker compose up autotests

Autotests depends on direct access to Keycloak API, so existing realm should be recreated.
As alternative the Direct access grants checkbox may be checked at Keycloak client settings.

To prevent pulling a new image:

sudo docker compose up autotests --pull=never

Dev mode​

Synopsis​

Dev mode allows to build images instead of pulling pre-built ones, and also adds a few useful overrides.

Look at Dev options section of your example.env file copy.
To enable dev options uncomment required COMPOSE_FILE= lines. Then run compose as usual.

Use --watch flag or toggle watch with w in attached mode to rebuld dev-services on changes.

caution

Do not use development mode on production environments because it may change the data.

Build images in dev mode​

To build clean images use command with enabled COMPOSE_FILE overrides:

sudo docker compose build --no-cache

--no-cache guarantees build with latest dependencies.

Connect to redis-salt Redis instance​

Dev mode allows to connect to the Redis instance by URL rediss://localhost:6379 (SIC!).
Since TLS is enabled client may skip a certificate validation (option like --insecure) or use a CA certificate.
The last may be obtained with command:

sudo docker compose exec redis-salt cat /etc/redis/certs/ca.crt

Dev minions​

Dev mode provides amount of impersistent minions in replica mode.
Look for options in the example.env file.

Dev minions do not keep their keys between restarts so keys will be dropped on the master.
Some operations may result in the loss of minions. In that case try the following command, which should reconnect minions:

sudo docker compose restart salt-master
info

docker compose up --force-recreate salt-master command does not regenerate minions.

Cleanup​

Cleanup data​

After changes made, created containers and volumes may become incompatible with current code without special migrations.
To fix startup problems on developement environment stop containters with Control-c and make them down:

sudo docker compose -f compose.yaml -f compose-dev-override.yaml down --volumes
info

The --volumes flag will purge attached volumes and result in data loss. Make sure not to lose production data.

Cleanup stale Docker stuff​

When code and configs change, new layers and other objects are created. To free up resources, run the following command from time to time:

sudo docker system prune --force

This is usually safe and only removes outdated data.

Software development conventions​

Redis: channels​

Hash name should be in form of OBJ_TYPE:{ID}:DATA_TYPE e.g. minion:{MID}:grains.
Use the singular form for the OBJ_TYPE and the plural form for the DATA_TYPE because there are multiple values ​​for an object.

Keycloak​

Keycloak for admin user (password in file ./secrets/keycloak_admin_password): http://localhost/auth/keycloak/