ai-tax-agent/infra

AI Tax Agent Infrastructure

Multi-environment Docker Compose infrastructure for AI Tax Agent.

For local development use the dedicated self-signed stack in infra/compose (see infra/compose/README.md). For remote environments use the shared base files with infra/scripts/deploy.sh and the envs in infra/environments.

Directory Structure

infra/
├── environments/           # Environment-specific configurations
│   ├── local/             # Local development (localhost, self-signed certs)
│   ├── development/       # Development server (dev.harkon.co.uk)
│   └── production/        # Production server (harkon.co.uk)
│
├── base/                  # Base compose files (shared across environments)
│   ├── infrastructure.yaml    # Core infra (Vault, MinIO, DBs, etc.)
│   ├── monitoring.yaml        # Monitoring stack (Prometheus, Grafana, Loki)
│   ├── services.yaml          # Application services
│   └── external.yaml          # External services (Traefik, Authentik, Gitea, etc.)
│
├── configs/               # Service configurations
│   ├── traefik/          # Traefik configs
│   ├── grafana/          # Grafana dashboards & provisioning
│   ├── prometheus/       # Prometheus config
│   ├── loki/             # Loki config
│   ├── vault/            # Vault config
│   └── authentik/        # Authentik bootstrap
│
├── certs/                # SSL certificates (gitignored)
│   ├── local/           # Self-signed certs for local
│   ├── development/     # Let's Encrypt certs for dev
│   └── production/      # Let's Encrypt certs for prod
│
└── scripts/              # Deployment scripts
    ├── deploy.sh         # Main deployment script
    ├── setup-networks.sh # Create Docker networks
    └── cleanup.sh        # Cleanup script

Environments

Local Development

• Domain: localhost / *.local.harkon.co.uk
• SSL: Self-signed certificates
• Auth: Authentik (optional)
• Registry: Local Docker registry or Gitea
• Purpose: Local development and testing

Development

• Domain: *.dev.harkon.co.uk
• SSL: Let's Encrypt (DNS-01 challenge)
• Auth: Authentik SSO
• Registry: Gitea container registry
• Purpose: Staging/testing before production

Production

• Domain: *.harkon.co.uk
• SSL: Let's Encrypt (DNS-01 challenge)
• Auth: Authentik SSO
• Registry: Gitea container registry
• Purpose: Production deployment

Quick Start

1. Setup Environment

# Choose your environment
export ENV=local  # or development, production

# Copy environment template
cp infra/environments/$ENV/.env.example infra/environments/$ENV/.env

# Edit environment variables
vim infra/environments/$ENV/.env

2. Generate Secrets (Production/Development only)

./scripts/generate-production-secrets.sh

3. Create Docker Networks

./infra/scripts/setup-networks.sh

4. Deploy Infrastructure

# Deploy everything
./infra/scripts/deploy.sh $ENV all

# Or deploy specific stacks
./infra/scripts/deploy.sh $ENV infrastructure
./infra/scripts/deploy.sh $ENV monitoring
./infra/scripts/deploy.sh $ENV services

Environment Variables

Each environment has its own .env file with:

• Domain Configuration: DOMAIN, EMAIL
• Database Passwords: POSTGRES_PASSWORD, NEO4J_PASSWORD, etc.
• Object Storage: MINIO_ROOT_USER, MINIO_ROOT_PASSWORD
• Secrets Management: VAULT_DEV_ROOT_TOKEN_ID
• SSO/Auth: AUTHENTIK_SECRET_KEY, AUTHENTIK_BOOTSTRAP_PASSWORD
• Monitoring: GRAFANA_PASSWORD, OAuth secrets
• Application: Service-specific configs

Deployment Commands

Deploy Full Stack

# Local
./infra/scripts/deploy.sh local all

# Development
./infra/scripts/deploy.sh development all

# Production
./infra/scripts/deploy.sh production all

Deploy Individual Stacks

# Infrastructure only (Vault, MinIO, DBs, etc.)
./infra/scripts/deploy.sh production infrastructure

# Monitoring only (Prometheus, Grafana, Loki)
./infra/scripts/deploy.sh production monitoring

# Services only (Application microservices)
./infra/scripts/deploy.sh production services

# External services (Traefik, Authentik, Gitea - usually pre-existing)
./infra/scripts/deploy.sh production external

Stop/Remove Stacks

# Stop all
./infra/scripts/deploy.sh production down

# Stop specific stack
docker compose -f infra/base/infrastructure.yaml --env-file infra/environments/production/.env down

Network Architecture

All environments use two Docker networks:

• frontend: Public-facing services (Traefik, UI)
• backend: Internal services (DBs, message queues, etc.)

Networks are created with:

docker network create frontend
docker network create backend

Volume Management

Volumes are environment-specific and named with environment prefix:

• Local: local_postgres_data, local_vault_data, etc.
• Development: dev_postgres_data, dev_vault_data, etc.
• Production: prod_postgres_data, prod_vault_data, etc.

SSL Certificates

Local

• Self-signed certificates in infra/certs/local/
• Generated with scripts/generate-dev-certs.sh

Development/Production

• Let's Encrypt certificates via Traefik
• DNS-01 challenge using GoDaddy API
• Stored in infra/certs/{environment}/

External Services

Some services (Traefik, Authentik, Gitea, Nextcloud, Portainer) may already exist on the server.

To use existing services:

1. Don't deploy external.yaml
2. Ensure networks are shared
3. Update service discovery labels

Monitoring

Access monitoring dashboards:

• Grafana: https://grafana.{domain}
• Prometheus: https://prometheus.{domain}
• Traefik Dashboard: https://traefik.{domain}/dashboard/

Troubleshooting

Check Service Status

docker compose -f infra/base/infrastructure.yaml --env-file infra/environments/production/.env ps

View Logs

docker compose -f infra/base/infrastructure.yaml --env-file infra/environments/production/.env logs -f vault

Restart Service

docker compose -f infra/base/infrastructure.yaml --env-file infra/environments/production/.env restart vault

Security Notes

• Never commit .env files - They contain secrets!
• Rotate secrets regularly - Use generate-production-secrets.sh
• Use strong passwords - Minimum 32 characters
• Enable Authentik SSO - For all production services
• Backup volumes - Especially databases and Vault

Migration from Old Structure

If migrating from the old structure:

1. Copy environment variables from old .env files
2. Update volume names if needed
3. Migrate data volumes
4. Update Traefik labels if using existing Traefik
5. Test in development first!

Support

For issues or questions:

• Check logs: docker compose logs -f <service>
• Review documentation in docs/
• Check Traefik dashboard for routing issues