completed local setup with compose
Some checks failed
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled

This commit is contained in:
harkon
2025-11-26 13:17:17 +00:00
parent 8fe5e62fee
commit fdba81809f
87 changed files with 5610 additions and 3376 deletions

View File

@@ -1,133 +1,23 @@
# External Services
# Compose Stacks
This directory contains Docker Compose configurations for external services that run on the production server.
This folder is for the self-contained local stack (self-signed TLS) and Traefik assets. Remote environments use the shared compose files in `infra/base` together with `infra/scripts/deploy.sh`.
## Services
## Local development (self-signed TLS)
- Copy envs: `cp infra/compose/env.example infra/compose/.env` then set passwords/secrets and the dev domain (defaults to `local.lan`).
- Host aliases: add the domain to `/etc/hosts` (e.g. `127.0.0.1 auth.local.lan api.local.lan grafana.local.lan vault.local.lan minio.local.lan`).
- Networks: `./infra/scripts/setup-networks.sh` (creates `apa-frontend` and `apa-backend` used everywhere).
- Run: `cd infra/compose && docker compose --env-file .env -f docker-compose.local.yml up -d`.
- Stop: `docker compose --env-file .env -f docker-compose.local.yml down`.
- TLS: Traefik mounts `infra/compose/traefik/certs/local.{crt,key}`. Regenerate if needed with `openssl req -x509 -newkey rsa:2048 -nodes -keyout infra/compose/traefik/certs/local.key -out infra/compose/traefik/certs/local.crt -days 365 -subj "/CN=*.local.lan"`.
### Traefik
- **Location**: `traefik/`
- **Purpose**: Reverse proxy and load balancer for all services
- **Deploy**: `cd traefik && docker compose up -d`
- **Access**: https://traefik.harkon.co.uk
## Cloud / remote (Lets Encrypt)
- Config lives in `infra/base` with env files in `infra/environments/{development,production}/.env`.
- Create the same docker networks on the host (`./infra/scripts/setup-networks.sh`) so Traefik and services share `apa-frontend` / `apa-backend`.
- Deploy on the server: `./infra/scripts/deploy.sh <environment> all` (or `infrastructure`, `monitoring`, `services`).
- Certificates: Traefik uses DNS-01 via GoDaddy from the provider env in `infra/base/traefik/config` (make sure `DOMAIN`, ACME email, and provider creds are set in the env file).
### Authentik
- **Location**: `authentik/`
- **Purpose**: SSO and authentication provider
- **Deploy**: `cd authentik && docker compose up -d`
- **Access**: https://authentik.harkon.co.uk
### Gitea
- **Location**: `gitea/`
- **Purpose**: Git repository hosting and container registry
- **Deploy**: `cd gitea && docker compose up -d`
- **Access**: https://gitea.harkon.co.uk
### Nextcloud
- **Location**: `nextcloud/`
- **Purpose**: File storage and collaboration
- **Deploy**: `cd nextcloud && docker compose up -d`
- **Access**: https://nextcloud.harkon.co.uk
### Portainer
- **Location**: `portainer/`
- **Purpose**: Docker management UI
- **Deploy**: `cd portainer && docker compose up -d`
- **Access**: https://portainer.harkon.co.uk
## Deployment
### Production (Remote Server)
```bash
# SSH to server
ssh deploy@141.136.35.199
# Navigate to service directory
cd /opt/ai-tax-agent/infra/compose/<service>
# Deploy service
docker compose up -d
# Check logs
docker compose logs -f
# Check status
docker compose ps
```
### Local Development
For local development, use the all-in-one compose file:
```bash
cd infra/compose
docker compose -f docker-compose.local.yml up -d
```
## Configuration
Each service has its own `.env` file for environment-specific configuration:
- `traefik/.provider.env` - GoDaddy API credentials
- `authentik/.env` - Authentik secrets
- `gitea/.env` - Gitea database credentials
## Networks
All services use shared Docker networks:
- `frontend` - Public-facing services
- `backend` - Internal services
Create networks before deploying:
```bash
docker network create frontend
docker network create backend
```
## Maintenance
### Update Service
```bash
cd /opt/ai-tax-agent/infra/compose/<service>
docker compose pull
docker compose up -d
```
### Restart Service
```bash
cd /opt/ai-tax-agent/infra/compose/<service>
docker compose restart
```
### View Logs
```bash
cd /opt/ai-tax-agent/infra/compose/<service>
docker compose logs -f
```
### Backup Data
```bash
# Backup volumes
docker run --rm -v <service>_data:/data -v $(pwd):/backup alpine tar czf /backup/<service>-backup.tar.gz /data
```
## Integration with Application
These external services are used by the application infrastructure:
- **Traefik** - Routes traffic to application services
- **Authentik** - Provides SSO for application UIs
- **Gitea** - Hosts Docker images for application services
The application infrastructure is deployed separately using:
```bash
./infra/scripts/deploy.sh production infrastructure
./infra/scripts/deploy.sh production services
```
## Files of note
- `docker-compose.local.yml` full local stack.
- `traefik/traefik.local.yml` and `traefik/traefik-dynamic.local.yml` static/dynamic Traefik config for local.
- `traefik/certs/` self-signed certs used by the local proxy.
- `env.example` defaults for local `.env`.