Files
ai-tax-agent/docs/VM.md
harkon b324ff09ef
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
Initial commit
2025-10-11 08:41:36 +01:00

306 lines
7.8 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# VM Setup
# 0) One-time VM prep (as root just this once)
SSH to the VM your provider gave you (often only root works initially):
```bash
ssh root@<VM_IP>
```
Create a non-root deploy user with sudo, and lock down SSH:
```bash
# create user
adduser deploy
usermod -aG sudo deploy
# add your SSH key
mkdir -p /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys # paste your public key
chmod 600 /home/deploy/.ssh/authorized_keys
chown -R deploy:deploy /home/deploy/.ssh
# harden SSH (optional but recommended)
sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
systemctl reload sshd
exit
```
Now reconnect as your non-root user:
```bash
ssh deploy@<VM_IP>
```
# 1) Firewall and basics
```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y ufw
# allow SSH + web
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status
```
# 2) Install Docker Engine + Compose plugin (non-root usage)
```bash
# Docker official repo
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu $(. /etc/os-release; echo $VERSION_CODENAME) stable" \
| sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# let your user run docker without sudo
sudo usermod -aG docker $USER
newgrp docker
# optional: limit container logs
echo '{"log-driver":"json-file","log-opts":{"max-size":"10m","max-file":"3"}}' | \
sudo tee /etc/docker/daemon.json
sudo systemctl restart docker
```
# 3) Layout for your Compose stacks
Well keep everything under `/opt/compose`, owned by `deploy`:
```bash
sudo mkdir -p /opt/compose/{traefik,portainer,gitea,authentik}
sudo chown -R deploy:deploy /opt/compose
```
Create the shared external Docker network (once):
```bash
docker network create proxy
```
# 4) Copy your compose files (no root, via scp/rsync)
From your **local** machine:
```bash
# example: copy a whole folder into /opt/compose/portainer
scp -r ./portainer/* deploy@<VM_IP>:/opt/compose/portainer/
# or use rsync (recommended)
rsync -avz ./gitea/ deploy@<VM_IP>:/opt/compose/gitea/
```
# 5) Traefik on the VM (HTTP-01 with Lets Encrypt)
On the VM:
```bash
cd /opt/compose/traefik
```
Create `compose.yml`:
```yaml
version: "3.9"
services:
traefik:
image: traefik:v3.1
restart: unless-stopped
command:
- --providers.docker=true
- --providers.docker.exposedByDefault=false
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.web.http.redirections.entryPoint.to=websecure
- --entrypoints.web.http.redirections.entryPoint.scheme=https
# Let's Encrypt (HTTP-01 challenge)
- --certificatesresolvers.le.acme.email=${LE_EMAIL}
- --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
- --certificatesresolvers.le.acme.httpchallenge=true
- --certificatesresolvers.le.acme.httpchallenge.entrypoint=web
# Optional dashboard (protect later)
- --api.dashboard=true
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./letsencrypt:/letsencrypt
networks:
- proxy
labels:
- traefik.enable=true
- traefik.http.routers.traefik.rule=Host(`traefik.YOURDOMAIN.com`)
- traefik.http.routers.traefik.entrypoints=websecure
- traefik.http.routers.traefik.tls.certresolver=le
- traefik.http.routers.traefik.service=api@internal
networks:
proxy:
external: true
```
Create the storage file and set strict perms:
```bash
mkdir -p /opt/compose/traefik/letsencrypt
touch /opt/compose/traefik/letsencrypt/acme.json
chmod 600 /opt/compose/traefik/letsencrypt/acme.json
```
Create `.env`:
```bash
echo "LE_EMAIL=you@example.com" > /opt/compose/traefik/.env
```
Bring it up:
```bash
cd /opt/compose/traefik
docker compose up -d
```
# 6) DNS records on GoDaddy
Point your domain/subdomains to the VMs **public IP**:
- `A @ -> <VM_IP>`
- `A traefik -> <VM_IP>`
- `A portainer -> <VM_IP>`
- `A git -> <VM_IP>`
- `A auth -> <VM_IP>`
(HTTP-01 will fetch per-host certs automatically the first time you visit each hostname.)
> If you want a **wildcard** (`*.example.com`), switch Traefik to **DNS-01** with your DNS providers API. GoDaddys API can be restrictive; moving DNS hosting to Cloudflare is common. But HTTP-01 works fine for named subdomains.
# 7) Example app stacks (all non-root)
## Portainer (behind Traefik)
`/opt/compose/portainer/compose.yml`
```yaml
version: "3.9"
services:
portainer:
image: portainer/portainer-ce:latest
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer_data:/data
networks:
- proxy
labels:
- traefik.enable=true
- traefik.http.routers.portainer.rule=Host(`portainer.YOURDOMAIN.com`)
- traefik.http.routers.portainer.entrypoints=websecure
- traefik.http.routers.portainer.tls.certresolver=le
- traefik.http.services.portainer.loadbalancer.server.port=9000
volumes:
portainer_data:
networks:
proxy:
external: true
```
Deploy:
```bash
cd /opt/compose/portainer
docker compose up -d
```
## Gitea (behind Traefik)
`/opt/compose/gitea/compose.yml`
```yaml
version: "3.9"
services:
gitea:
image: gitea/gitea:1
restart: unless-stopped
environment:
- USER_UID=1000
- USER_GID=1000
volumes:
- gitea_data:/data
networks:
- proxy
labels:
- traefik.enable=true
- traefik.http.routers.gitea.rule=Host(`git.YOURDOMAIN.com`)
- traefik.http.routers.gitea.entrypoints=websecure
- traefik.http.routers.gitea.tls.certresolver=le
- traefik.http.services.gitea.loadbalancer.server.port=3000
volumes:
gitea_data:
networks:
proxy:
external: true
```
(Do the same for Authentik; keep it on `proxy` and add Traefik labels to the web service.)
# 8) Secure the Traefik dashboard (quick basic-auth)
Create a middleware once and attach it to the dashboard router.
Generate a bcrypt hash (on your laptop):
```bash
# Install apache2-utils if you have it, or use Docker to generate:
docker run --rm httpd:2.4-alpine htpasswd -nbB admin 'YOUR_STRONG_PASSWORD'
# Output looks like: admin:$2y$05$....
```
Add to Traefik labels:
```yaml
labels:
- traefik.enable=true
- traefik.http.middlewares.basicauth.basicauth.users=admin:$$2y$$05$$<HASH_REST>
- traefik.http.routers.traefik.rule=Host(`traefik.YOURDOMAIN.com`)
- traefik.http.routers.traefik.entrypoints=websecure
- traefik.http.routers.traefik.tls.certresolver=le
- traefik.http.routers.traefik.middlewares=basicauth@docker
- traefik.http.routers.traefik.service=api@internal
```
Then:
```bash
cd /opt/compose/traefik && docker compose up -d
```
# 9) Quality-of-life tips
- Containers should include `restart: unless-stopped`; Docker will auto-start them on reboot—no systemd unit needed.
- Keep everything on the `proxy` network; only Traefik publishes 80/443 to the host.
- For updates: `docker compose pull && docker compose up -d` per stack.
- Backups: snapshot `/opt/compose/*` and any named volumes (`/var/lib/docker/volumes/...`), or mount volumes to known paths you can back up.
---
If you want, paste your existing Traefik/Authentik/Gitea labels here and Ill adapt them for the VM layout (and wire Authentik as forward-auth to protect Portainer/Gitea).