refactor(server-tools): drop systemd and overlay management from ansible (DEVOPS-665)

server-tools/README.md

@@ -1,12 +1,12 @@
 # DHIS2 Docker Deployment Ansible Playbook
 
-This Ansible playbook automates the deployment of the DHIS2 Docker stack.
+This Ansible playbook provisions a host for running the DHIS2 Docker stack. It installs Docker, applies firewall and security hardening, and clones this repository onto the host. Starting the stack itself is done manually with the `make start-*` targets from the cloned working tree, not by Ansible.
 
 ## Features
 
 - **Infrastructure Bootstrapping**: Installs required system packages including Docker and Docker Compose
 - **Security Hardening**: Applies system hardening based on the microk8s-playbook harden.yaml, adapted for Docker
-- **Deployment Automation**: Clones the repository, generates .env file, and deploys with selected overlays
+- **Repository Checkout**: Clones the deployment repository to `deploy_dir`
 - **Modularity**: Uses Ansible roles for easy extension and maintenance
 - **Idempotency**: Safe to run multiple times
 
@@ -15,65 +15,55 @@ This Ansible playbook automates the deployment of the DHIS2 Docker stack.
 - Ansible installed on the control machine
 - Target server with Ubuntu 24.04
 - SSH access to the target server with sudo privileges
-- Environment variables set: `GEN_APP_HOSTNAME` and `GEN_LETSENCRYPT_ACME_EMAIL`
 
 ## Configuration
 
 Edit `group_vars/all.yml` to customize:
 
-- `app_hostname`: Application hostname (set via env var)
-- `letsencrypt_email`: Let's Encrypt email (set via env var)
-- `overlays`: List of overlays to enable, e.g., `['monitoring']`
-- Other variables as needed
+- `repo_url`: Repository to clone onto the host
+- `deploy_dir`: Where to clone it
+- `firewall_allowed_ports`: Host-facing TCP ports to open
+- `allowed_ssh_users`: Users allowed to SSH in
+- `docker_user` / `docker_group` / `docker_home`: Account that owns the Docker workload
 
 ## Usage
 
-1. Set environment variables:
+1. Update the inventory file `inventory.ini` according to your needs
 
-    ```bash
-    export GEN_APP_HOSTNAME=your.domain.com
-    export GEN_LETSENCRYPT_ACME_EMAIL=your.email@example.com
-    ```
-
-2. Update the inventory file `inventory.ini` according to your needs
-
-3. Copy your public SSH key to the target server
+2. Copy your public SSH key to the target server
 
     ```bash
     ssh-copy-id ubuntu@<your server ip>
     ```
 
-4. Store your user's sudo password in `./.ansible_become_pass`
+3. Store your user's sudo password in `./.ansible_become_pass`
 
-5. Run the playbook:
+4. Run the playbook:
 
     ```bash
     make deployment
     ```
 
+5. Once provisioning finishes, SSH to the host and start the stacks from the cloned repo using the `make start-*` targets (see the repo's top-level README).
+
 ## Roles
 
 - **bootstrap**: Installs Docker, creates users, sets up directories
 - **firewall**: Configures firewall rules for Docker and host-facing ports
 - **harden**: Applies security hardening (SSH, kernel, Docker config)
-- **deploy**: Clones repo, generates .env, runs docker-compose
+- **deploy**: Clones the deployment repository to `deploy_dir`
 
 ## Security Notes
 
 - Docker is configured with user namespace remapping for least privilege
 - Firewall rules are configured to deny all by default and only allow SSH, HTTP, HTTPS and inter-container communication only on default subnets
 - AppArmor is enabled
 - Unattended-upgrades are enabled
-- Secrets are handled via environment variables and .env file
 
-The above is only a subset of the security hardening that is applied. For more information, see
-the [firewall](roles/firewall/tasks/main.yml) and [harden](roles/harden/tasks/main.yml) roles.
+The above is only a subset of the security hardening that is applied. For more information, see the [firewall](roles/firewall/tasks/main.yml) and [harden](roles/harden/tasks/main.yml) roles.
 
 ### Firewall Management
 
-All firewall rules for Docker and host-facing ports are managed by the `firewall` role.
-See [roles/firewall/tasks/main.yml](roles/firewall/tasks/main.yml) for more details.
+All firewall rules for Docker and host-facing ports are managed by the `firewall` role. See [roles/firewall/tasks/main.yml](roles/firewall/tasks/main.yml) for more details.
 
-⚠️ **Important:** Do **not** use UFW or other firewall frontends alongside this setup. Docker bypasses standard host
-chains (INPUT/OUTPUT), so UFW rules are ignored or may conflict. All host and container traffic should be managed
-exclusively through this Ansible role.
+⚠️ **Important:** Do **not** use UFW or other firewall frontends alongside this setup. Docker bypasses standard host chains (INPUT/OUTPUT), so UFW rules are ignored or may conflict. All host and container traffic should be managed exclusively through this Ansible role.

server-tools/group_vars/all.yml

@@ -1,19 +1,7 @@
-# Configuration variables for the DHIS2 Docker deployment playbook
-
-# Application hostname (required for .env generation)
-app_hostname: "{{ lookup('env', 'GEN_APP_HOSTNAME') | mandatory }}"
-
-# Let's Encrypt ACME email (required for .env generation)
-letsencrypt_email: "{{ lookup('env', 'GEN_LETSENCRYPT_ACME_EMAIL') | mandatory }}"
-
-# List of overlays to enable (e.g., ['monitoring'])
-# Note that if you add or remove overlays, all services will be stopped and restarted, so there will be some downtime
-overlays: [ ]
+# Configuration variables for the DHIS2 Docker server provisioning playbook
 
 repo_url: https://github.com/dhis2/docker-deployment
 
-repo_branch: master
-
 deploy_dir: /opt/dhis2
 
 firewall_allowed_ports: [ 22, 80, 443 ]
@@ -26,6 +14,3 @@ docker_user: dhis2
 docker_group: docker
 
 docker_home: "/home/{{ docker_user }}"
-
-# Enable deployment of DHIS2
-enable_deploy: true

server-tools/roles/deploy/tasks/main.yml

@@ -1,196 +1,11 @@
-- name: Check if deploy dir exists
-  stat:
-    path: "{{ deploy_dir }}"
-  register: dir_stat
-
 - name: Check if deploy dir is a git repo
   stat:
     path: "{{ deploy_dir }}/.git"
   register: git_repo
 
-- name: Check deploy dir contents
-  find:
-    path: "{{ deploy_dir }}"
-    file_type: any
-  register: dir_contents
-  when: dir_stat.stat.exists
-
-- name: Fail if deploy dir exists, is not empty, and is not a git repo
-  fail:
-    msg: "Deploy directory {{ deploy_dir }} exists, is not empty, and is not a git repository. Please remove it manually or check for conflicts."
-  when: dir_stat.stat.exists and not git_repo.stat.exists and dir_contents.files | length > 0
-
 - name: Clone the repository
   command: git clone {{ repo_url }} {{ deploy_dir }}
   when: not git_repo.stat.exists
 
 - name: Add deploy directory to git safe directories
   command: git config --global --add safe.directory {{ deploy_dir }}
-
-- name: Fetch all branches
-  command: git fetch --all
-  args:
-    chdir: "{{ deploy_dir }}"
-
-- name: Checkout the specified branch
-  command: git checkout {{ repo_branch }}
-  args:
-    chdir: "{{ deploy_dir }}"
-
-- name: Pull latest changes
-  command: git pull
-  args:
-    chdir: "{{ deploy_dir }}"
-  when: git_repo.stat.exists
-
-- name: Generate .env file
-  shell: |
-    export GEN_APP_HOSTNAME="{{ app_hostname }}"
-    export GEN_LETSENCRYPT_ACME_EMAIL="{{ letsencrypt_email }}"
-    ./scripts/generate-env.sh
-  args:
-    chdir: "{{ deploy_dir }}"
-    creates: "{{ deploy_dir }}/.env"
-
-- name: Set .env permissions
-  file:
-    path: "{{ deploy_dir }}/.env"
-    owner: "{{ docker_user }}"
-    group: "root"
-
-- name: Create Traefik acme.json
-  file:
-    path: "{{ deploy_dir }}/traefik/acme.json"
-    state: touch
-    owner: "65534"
-    group: "65534"
-    mode: 0600
-
-- name: Install Loki Docker driver
-  become: yes
-  command: ./scripts/install-loki-driver.sh
-  args:
-    chdir: "{{ deploy_dir }}"
-  when: "'monitoring' in overlays"
-
-- name: Read current overlays
-  slurp:
-    src: "{{ deploy_dir }}/current_overlays.json"
-  register: current_overlays_slurp
-  ignore_errors: true
-
-- name: Set old overlays
-  set_fact:
-    old_overlays: "{{ (current_overlays_slurp.content | b64decode | from_json) if current_overlays_slurp is succeeded else [] }}"
-
-- debug:
-    msg: "old_overlays: {{ old_overlays }}, overlays: {{ overlays }}"
-
-- name: Set compose files
-  set_fact:
-    compose_files: "{{ ['-f docker-compose.yml'] + overlays | map('regex_replace', '^(.*)$', '-f overlays/\\1/docker-compose.yml') | list }}"
-
-- debug:
-    msg: "compose_files: {{ compose_files }}"
-
-- debug:
-    msg: "overlays changed: {{ old_overlays != overlays }}"
-
-- name: Stop dhis2 service when overlays changed
-  systemd:
-    name: dhis2
-    state: stopped
-  when: old_overlays != overlays
-
-- name: Write current overlays
-  copy:
-    dest: "{{ deploy_dir }}/current_overlays.json"
-    content: "{{ overlays | to_json }}"
-
-- name: Create systemd service for dhis2
-  copy:
-    dest: /etc/systemd/system/dhis2.service
-    content: |
-      [Unit]
-      Description=DHIS2 Docker Compose Service
-      After=docker.service
-      Requires=docker.service
-
-      [Service]
-      Type=simple
-      User={{ docker_user }}
-      Group={{ docker_group }}
-      WorkingDirectory={{ deploy_dir }}
-      ExecStart=/usr/bin/docker compose {{ compose_files | join(' ') }} up
-      ExecStop=/usr/bin/docker compose {{ compose_files | join(' ') }} down
-      Restart=always
-
-      [Install]
-      WantedBy=multi-user.target
-
-- name: Show systemd service content
-  command: cat /etc/systemd/system/dhis2.service
-  register: systemd_content
-
-- debug:
-    msg: "{{ systemd_content.stdout }}"
-
-- name: Reload systemd daemon
-  systemd:
-    daemon_reload: yes
-
-- name: Enable dhis2 service
-  systemd:
-    name: dhis2
-    enabled: yes
-
-- name: Find all overlay directories
-  shell: find overlays/ -name "docker-compose.yml" -o -name "docker-compose.yaml" | xargs dirname | sed 's|overlays/||' | sort | uniq
-  args:
-    chdir: "{{ deploy_dir }}"
-  register: all_overlays_raw
-
-- name: Set all overlays
-  set_fact:
-    all_overlays: "{{ all_overlays_raw.stdout_lines | select | list }}"
-
-- name: Set all compose files
-  set_fact:
-    all_compose_files: "{{ ['-f docker-compose.yml'] + all_overlays | map('regex_replace', '^(.*)$', '-f overlays/\\1/docker-compose.yml') | list }}"
-
-- debug:
-    msg: "all_overlays: {{ all_overlays }}, all_compose_files: {{ all_compose_files }}"
-
-- debug:
-    msg: "Running: docker compose {{ all_compose_files | join(' ') }} down"
-
-- name: Down all services
-  command: docker compose {{ all_compose_files | join(' ') }} down
-  args:
-    chdir: "{{ deploy_dir }}"
-
-- debug:
-    msg: "Starting: docker compose {{ compose_files | join(' ') }} up"
-
-- debug:
-    msg: "old_overlays: {{ old_overlays }}, overlays: {{ overlays }}, changed: {{ old_overlays != overlays }}"
-
-- name: Start dhis2 service
-  systemd:
-    name: dhis2
-    state: started
-
-- name: Wait for services to be healthy
-  command: docker compose {{ compose_files | join(' ') }} ps
-  args:
-    chdir: "{{ deploy_dir }}"
-  register: compose_ps
-  # TODO: This doesn't work at all... if a container goes to unhealthy state this will pass
-  until: "'(healthy)' in compose_ps.stdout and 'starting' not in compose_ps.stdout and 'unhealthy' not in compose_ps.stdout"
-  retries: 30
-  delay: 10
-
-- name: Verify all services are running
-  assert:
-    that: "'Exit' not in compose_ps.stdout"
-    fail_msg: "Some services failed to start"

server-tools/site.yml

@@ -1,10 +1,9 @@
-- name: Deploy DHIS2 Docker Stack
+- name: Provision DHIS2 Docker host
   hosts: all
   become: true
 
   roles:
     - bootstrap
     - firewall
     - harden
-    - role: deploy
-      when: enable_deploy | default(false) | bool
+    - deploy