Files
ai-tax-agent/docs/VM.md
harkon b324ff09ef
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
Initial commit
2025-10-11 08:41:36 +01:00

7.8 KiB
Raw Permalink Blame History

VM Setup

0) One-time VM prep (as root just this once)

SSH to the VM your provider gave you (often only root works initially):

ssh root@<VM_IP>

Create a non-root deploy user with sudo, and lock down SSH:

# create user
adduser deploy
usermod -aG sudo deploy

# add your SSH key
mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys   # paste your public key
chmod 600 /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh

# harden SSH (optional but recommended)
sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
systemctl reload sshd
exit

Now reconnect as your non-root user:

ssh deploy@<VM_IP>

1) Firewall and basics

# Ubuntu/Debian
sudo apt update
sudo apt install -y ufw

# allow SSH + web
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status

2) Install Docker Engine + Compose plugin (non-root usage)

# Docker official repo
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" \
  | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# let your user run docker without sudo
sudo usermod -aG docker $USER
newgrp docker

# optional: limit container logs
echo '{"log-driver":"json-file","log-opts":{"max-size":"10m","max-file":"3"}}' | \
  sudo tee /etc/docker/daemon.json
sudo systemctl restart docker

3) Layout for your Compose stacks

Well keep everything under /opt/compose, owned by deploy:

sudo mkdir -p /opt/compose/{traefik,portainer,gitea,authentik}
sudo chown -R deploy:deploy /opt/compose

Create the shared external Docker network (once):

docker network create proxy

4) Copy your compose files (no root, via scp/rsync)

From your local machine:

# example: copy a whole folder into /opt/compose/portainer
scp -r ./portainer/* deploy@<VM_IP>:/opt/compose/portainer/

# or use rsync (recommended)
rsync -avz ./gitea/ deploy@<VM_IP>:/opt/compose/gitea/

5) Traefik on the VM (HTTP-01 with Lets Encrypt)

On the VM:

cd /opt/compose/traefik

Create compose.yml:

version: "3.9"
services:
  traefik:
    image: traefik:v3.1
    restart: unless-stopped
    command:
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https

      # Let's Encrypt (HTTP-01 challenge)
      - --certificatesresolvers.le.acme.email=${LE_EMAIL}
      - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
      - --certificatesresolvers.le.acme.httpchallenge=true
      - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web

      # Optional dashboard (protect later)
      - --api.dashboard=true
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./letsencrypt:/letsencrypt
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik.rule=Host(`traefik.YOURDOMAIN.com`)
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.tls.certresolver=le
      - traefik.http.routers.traefik.service=api@internal

networks:
  proxy:
    external: true

Create the storage file and set strict perms:

mkdir -p /opt/compose/traefik/letsencrypt
touch /opt/compose/traefik/letsencrypt/acme.json
chmod 600 /opt/compose/traefik/letsencrypt/acme.json

Create .env:

echo "LE_EMAIL=you@example.com" > /opt/compose/traefik/.env

Bring it up:

cd /opt/compose/traefik
docker compose up -d

6) DNS records on GoDaddy

Point your domain/subdomains to the VMs public IP:

  • A @ -> <VM_IP>
  • A traefik -> <VM_IP>
  • A portainer -> <VM_IP>
  • A git -> <VM_IP>
  • A auth -> <VM_IP>

(HTTP-01 will fetch per-host certs automatically the first time you visit each hostname.)

If you want a wildcard (*.example.com), switch Traefik to DNS-01 with your DNS providers API. GoDaddys API can be restrictive; moving DNS hosting to Cloudflare is common. But HTTP-01 works fine for named subdomains.

7) Example app stacks (all non-root)

Portainer (behind Traefik)

/opt/compose/portainer/compose.yml

version: "3.9"
services:
  portainer:
    image: portainer/portainer-ce:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.portainer.rule=Host(`portainer.YOURDOMAIN.com`)
      - traefik.http.routers.portainer.entrypoints=websecure
      - traefik.http.routers.portainer.tls.certresolver=le
      - traefik.http.services.portainer.loadbalancer.server.port=9000

volumes:
  portainer_data:

networks:
  proxy:
    external: true

Deploy:

cd /opt/compose/portainer
docker compose up -d

Gitea (behind Traefik)

/opt/compose/gitea/compose.yml

version: "3.9"
services:
  gitea:
    image: gitea/gitea:1
    restart: unless-stopped
    environment:
      - USER_UID=1000
      - USER_GID=1000
    volumes:
      - gitea_data:/data
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.gitea.rule=Host(`git.YOURDOMAIN.com`)
      - traefik.http.routers.gitea.entrypoints=websecure
      - traefik.http.routers.gitea.tls.certresolver=le
      - traefik.http.services.gitea.loadbalancer.server.port=3000

volumes:
  gitea_data:

networks:
  proxy:
    external: true

(Do the same for Authentik; keep it on proxy and add Traefik labels to the web service.)

8) Secure the Traefik dashboard (quick basic-auth)

Create a middleware once and attach it to the dashboard router.

Generate a bcrypt hash (on your laptop):

# Install apache2-utils if you have it, or use Docker to generate:
docker run --rm httpd:2.4-alpine htpasswd -nbB admin 'YOUR_STRONG_PASSWORD'
# Output looks like: admin:$2y$05$....

Add to Traefik labels:

labels:
  - traefik.enable=true
  - traefik.http.middlewares.basicauth.basicauth.users=admin:$$2y$$05$$<HASH_REST>
  - traefik.http.routers.traefik.rule=Host(`traefik.YOURDOMAIN.com`)
  - traefik.http.routers.traefik.entrypoints=websecure
  - traefik.http.routers.traefik.tls.certresolver=le
  - traefik.http.routers.traefik.middlewares=basicauth@docker
  - traefik.http.routers.traefik.service=api@internal

Then:

cd /opt/compose/traefik && docker compose up -d

9) Quality-of-life tips

  • Containers should include restart: unless-stopped; Docker will auto-start them on reboot—no systemd unit needed.
  • Keep everything on the proxy network; only Traefik publishes 80/443 to the host.
  • For updates: docker compose pull && docker compose up -d per stack.
  • Backups: snapshot /opt/compose/* and any named volumes (/var/lib/docker/volumes/...), or mount volumes to known paths you can back up.

If you want, paste your existing Traefik/Authentik/Gitea labels here and Ill adapt them for the VM layout (and wire Authentik as forward-auth to protect Portainer/Gitea).