feat: working infra with sso
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
This commit is contained in:
21
README.md
21
README.md
@@ -156,7 +156,7 @@ The deployment automation handles:
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
ai-tax-agent-2/
|
||||
ai-tax-agent/
|
||||
├── libs/ # Shared libraries
|
||||
│ ├── config.py # Configuration and factories
|
||||
│ ├── security.py # Authentication and encryption
|
||||
@@ -356,20 +356,17 @@ make clean # Clean up volumes and networks
|
||||
|
||||
### Production Deployment
|
||||
|
||||
For detailed instructions, see [infra/compose/README.md](infra/compose/README.md).
|
||||
|
||||
The system uses a unified deployment script for production environments:
|
||||
|
||||
```bash
|
||||
# Using Docker Swarm
|
||||
make deploy-swarm
|
||||
|
||||
# Using Kubernetes
|
||||
make deploy-k8s
|
||||
|
||||
# Using Terraform (AWS/Azure/GCP)
|
||||
cd infra/terraform
|
||||
terraform init
|
||||
terraform plan
|
||||
terraform apply
|
||||
# Deploy to production (Infrastructure + Services + Monitoring)
|
||||
./infra/scripts/deploy.sh production all
|
||||
```
|
||||
|
||||
Ensure you have configured `infra/environments/production/.env` with the correct secrets and domain settings before deploying.
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
Key environment variables:
|
||||
|
||||
@@ -1,85 +0,0 @@
|
||||
# Authentik Configuration - Grafana SSO Integration
|
||||
# Generated: 2025-09-20 07:25:00
|
||||
# This file contains the Authentik configuration for Grafana OAuth2/OIDC integration
|
||||
# Apply this blueprint to automate the setup of Grafana SSO with Authentik
|
||||
|
||||
version: 1
|
||||
|
||||
metadata:
|
||||
name: AI Tax Agent Grafana SSO Integration
|
||||
labels:
|
||||
blueprints.goauthentik.io/generated: "true"
|
||||
|
||||
entries:
|
||||
# Grafana OAuth2 Provider
|
||||
- attrs:
|
||||
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
|
||||
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
|
||||
name: grafana
|
||||
client_type: confidential
|
||||
client_id: grafana
|
||||
client_secret: ${AUTHENTIK_GRAFANA_CLIENT_SECRET:-changeme-grafana-secret}
|
||||
redirect_uris:
|
||||
- https://grafana.${DOMAIN:-local.lan}/login/generic_oauth
|
||||
sub_mode: hashed_user_id
|
||||
include_claims_in_id_token: true
|
||||
issuer_mode: per_provider
|
||||
signing_key: !Find [authentik_crypto.certificatekeypair, [name, authentik Self-signed Certificate]]
|
||||
property_mappings:
|
||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, openid]]
|
||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, email]]
|
||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, profile]]
|
||||
- !KeyOf grafana-groups-mapping
|
||||
conditions: []
|
||||
identifiers:
|
||||
name: grafana
|
||||
model: authentik_providers_oauth2.oauth2provider
|
||||
permissions: []
|
||||
state: present
|
||||
|
||||
# Custom Groups Mapping for Grafana
|
||||
- attrs:
|
||||
name: Grafana Groups Mapping
|
||||
description: Maps Authentik groups to Grafana roles
|
||||
scope_name: groups
|
||||
expression: |
|
||||
# Map Authentik groups to Grafana roles
|
||||
groups = []
|
||||
user_groups = [group.name for group in request.user.ak_groups.all()]
|
||||
|
||||
# Admin role mapping
|
||||
if "authentik Admins" in user_groups or "Administrators" in user_groups:
|
||||
groups.append("Admin")
|
||||
|
||||
# Editor role mapping
|
||||
if "Tax Reviewers" in user_groups or "Accountants" in user_groups:
|
||||
groups.append("Editor")
|
||||
|
||||
# Viewer role mapping (default for all authenticated users)
|
||||
groups.append("Viewer")
|
||||
|
||||
return {
|
||||
"groups": groups,
|
||||
"role": groups[0] if groups else "Viewer" # Primary role
|
||||
}
|
||||
conditions: []
|
||||
identifiers:
|
||||
name: Grafana Groups Mapping
|
||||
model: authentik_providers_oauth2.scopemapping
|
||||
permissions: []
|
||||
state: present
|
||||
|
||||
# Grafana Application
|
||||
- attrs:
|
||||
name: Grafana
|
||||
slug: grafana
|
||||
provider: !KeyOf grafana
|
||||
policy_engine_mode: any
|
||||
meta_description: Grafana monitoring and observability platform
|
||||
meta_publisher: Grafana Labs
|
||||
conditions: []
|
||||
identifiers:
|
||||
slug: grafana
|
||||
model: authentik_core.application
|
||||
permissions: []
|
||||
state: present
|
||||
@@ -1,109 +0,0 @@
|
||||
# Simple Authentik Bootstrap Configuration
|
||||
# This file configures the basic Authentik setup for AI Tax Agent
|
||||
|
||||
version: 1
|
||||
|
||||
metadata:
|
||||
name: AI Tax Agent Simple Bootstrap
|
||||
|
||||
entries:
|
||||
# Create admin user
|
||||
- model: authentik_core.user
|
||||
identifiers:
|
||||
username: admin
|
||||
attrs:
|
||||
name: "System Administrator"
|
||||
email: admin@local.lan
|
||||
is_active: true
|
||||
is_staff: true
|
||||
is_superuser: true
|
||||
|
||||
# Create user groups
|
||||
- model: authentik_core.group
|
||||
identifiers:
|
||||
name: "Administrators"
|
||||
attrs:
|
||||
is_superuser: true
|
||||
|
||||
- model: authentik_core.group
|
||||
identifiers:
|
||||
name: "Tax Reviewers"
|
||||
attrs:
|
||||
is_superuser: false
|
||||
|
||||
- model: authentik_core.group
|
||||
identifiers:
|
||||
name: "Accountants"
|
||||
attrs:
|
||||
is_superuser: false
|
||||
|
||||
- model: authentik_core.group
|
||||
identifiers:
|
||||
name: "Clients"
|
||||
attrs:
|
||||
is_superuser: false
|
||||
|
||||
# Create OIDC Provider for API services
|
||||
- model: authentik_providers_oauth2.oauth2provider
|
||||
identifiers:
|
||||
name: "AI Tax Agent API"
|
||||
attrs:
|
||||
client_id: "ai-tax-agent-api"
|
||||
client_secret: !Env [AUTHENTIK_API_CLIENT_SECRET, "changeme-api-secret"]
|
||||
authorization_grant_type: "authorization-code"
|
||||
client_type: "confidential"
|
||||
redirect_uris: "https://api.local/auth/callback\nhttps://review.local/auth/callback"
|
||||
sub_mode: "hashed_user_id"
|
||||
include_claims_in_id_token: true
|
||||
issuer_mode: "per_provider"
|
||||
authorization_flow:
|
||||
!Find [authentik_flows.flow, [slug, "default-authentication-flow"]]
|
||||
invalidation_flow:
|
||||
!Find [authentik_flows.flow, [slug, "default-invalidation-flow"]]
|
||||
|
||||
# Create OIDC Provider for Grafana
|
||||
- model: authentik_providers_oauth2.oauth2provider
|
||||
identifiers:
|
||||
name: "Grafana"
|
||||
attrs:
|
||||
client_id: "grafana"
|
||||
client_secret:
|
||||
!Env [AUTHENTIK_GRAFANA_CLIENT_SECRET, "changeme-grafana-secret"]
|
||||
authorization_grant_type: "authorization-code"
|
||||
client_type: "confidential"
|
||||
redirect_uris: "https://grafana.local/login/generic_oauth"
|
||||
sub_mode: "hashed_user_id"
|
||||
include_claims_in_id_token: true
|
||||
issuer_mode: "per_provider"
|
||||
authorization_flow:
|
||||
!Find [authentik_flows.flow, [slug, "default-authentication-flow"]]
|
||||
invalidation_flow:
|
||||
!Find [authentik_flows.flow, [slug, "default-invalidation-flow"]]
|
||||
|
||||
# Create Applications
|
||||
- model: authentik_core.application
|
||||
identifiers:
|
||||
name: "AI Tax Agent API"
|
||||
slug: "ai-tax-agent-api"
|
||||
attrs:
|
||||
provider:
|
||||
!Find [
|
||||
authentik_providers_oauth2.oauth2provider,
|
||||
[name, "AI Tax Agent API"],
|
||||
]
|
||||
meta_launch_url: "https://api.local"
|
||||
meta_description: "AI Tax Agent API Services"
|
||||
meta_publisher: "AI Tax Agent"
|
||||
policy_engine_mode: "any"
|
||||
|
||||
- model: authentik_core.application
|
||||
identifiers:
|
||||
name: "Grafana"
|
||||
slug: "grafana"
|
||||
attrs:
|
||||
provider:
|
||||
!Find [authentik_providers_oauth2.oauth2provider, [name, "Grafana"]]
|
||||
meta_launch_url: "https://grafana.local"
|
||||
meta_description: "Monitoring and Observability Dashboard"
|
||||
meta_publisher: "AI Tax Agent"
|
||||
policy_engine_mode: "any"
|
||||
@@ -167,6 +167,7 @@ entries:
|
||||
- !KeyOf scope_email
|
||||
- !KeyOf scope_groups
|
||||
- !KeyOf scope_offline
|
||||
- !KeyOf scope_minio_policy
|
||||
authorization_flow: !KeyOf default_authz_flow
|
||||
invalidation_flow: !KeyOf default_inval_flow
|
||||
|
||||
@@ -258,6 +259,31 @@ entries:
|
||||
# Default to Viewer role
|
||||
return "Viewer"
|
||||
|
||||
# Custom Scope Mapping for MinIO
|
||||
- id: scope_minio_policy
|
||||
model: authentik_providers_oauth2.scopemapping
|
||||
state: present
|
||||
identifiers:
|
||||
name: "MinIO Policy Mapping"
|
||||
attrs:
|
||||
name: "MinIO Policy Mapping"
|
||||
description: "Maps Authentik groups to MinIO policies"
|
||||
scope_name: "minio"
|
||||
expression: |
|
||||
# Map Authentik groups to MinIO policies
|
||||
user_groups = [group.name for group in request.user.ak_groups.all()]
|
||||
|
||||
# Admin policy mapping
|
||||
if "Administrators" in user_groups:
|
||||
return {"policy": ["consoleAdmin"]}
|
||||
|
||||
# Default to readwrite for other known groups
|
||||
if "Tax Reviewers" in user_groups or "Accountants" in user_groups:
|
||||
return {"policy": ["readwrite"]}
|
||||
|
||||
# Default fallback
|
||||
return {"policy": ["readonly"]}
|
||||
|
||||
- model: authentik_providers_oauth2.oauth2provider
|
||||
state: present
|
||||
identifiers:
|
||||
@@ -340,3 +366,78 @@ entries:
|
||||
target: !Find [authentik_core.application, [slug, "grafana-prod"]]
|
||||
attrs:
|
||||
order: 0
|
||||
|
||||
# --- Qdrant (Production) ---------------------------------------------------
|
||||
- id: provider_qdrant
|
||||
model: authentik_providers_proxy.proxyprovider
|
||||
state: present
|
||||
identifiers:
|
||||
name: "Qdrant (Production)"
|
||||
attrs:
|
||||
external_host: "https://qdrant.app.harkon.co.uk:8444"
|
||||
authorization_flow: !KeyOf default_authz_flow
|
||||
invalidation_flow: !KeyOf default_inval_flow
|
||||
mode: "forward_single"
|
||||
|
||||
- model: authentik_core.application
|
||||
state: present
|
||||
identifiers:
|
||||
slug: "qdrant-prod"
|
||||
attrs:
|
||||
name: "Qdrant (Production)"
|
||||
provider: !KeyOf provider_qdrant
|
||||
meta_launch_url: "https://qdrant.app.harkon.co.uk:8444/dashboard"
|
||||
meta_description: "Vector Database Dashboard"
|
||||
meta_publisher: "AI Tax Agent"
|
||||
policy_engine_mode: "any"
|
||||
|
||||
# --- Neo4j (Production) ----------------------------------------------------
|
||||
- id: provider_neo4j
|
||||
model: authentik_providers_proxy.proxyprovider
|
||||
state: present
|
||||
identifiers:
|
||||
name: "Neo4j (Production)"
|
||||
attrs:
|
||||
external_host: "https://neo4j.app.harkon.co.uk:8444"
|
||||
authorization_flow: !KeyOf default_authz_flow
|
||||
invalidation_flow: !KeyOf default_inval_flow
|
||||
mode: "forward_single"
|
||||
|
||||
- model: authentik_core.application
|
||||
state: present
|
||||
identifiers:
|
||||
slug: "neo4j-prod"
|
||||
attrs:
|
||||
name: "Neo4j (Production)"
|
||||
provider: !KeyOf provider_neo4j
|
||||
meta_launch_url: "https://neo4j.app.harkon.co.uk:8444"
|
||||
meta_description: "Knowledge Graph Browser"
|
||||
meta_publisher: "AI Tax Agent"
|
||||
policy_engine_mode: "any"
|
||||
|
||||
# --- Policy Bindings for New Apps ------------------------------------------
|
||||
- model: authentik_policies.policybinding
|
||||
state: present
|
||||
identifiers:
|
||||
policy: !KeyOf policy_always_allow
|
||||
target: !Find [authentik_core.application, [slug, "qdrant-prod"]]
|
||||
attrs:
|
||||
order: 0
|
||||
|
||||
- model: authentik_policies.policybinding
|
||||
state: present
|
||||
identifiers:
|
||||
policy: !KeyOf policy_always_allow
|
||||
target: !Find [authentik_core.application, [slug, "neo4j-prod"]]
|
||||
attrs:
|
||||
order: 0
|
||||
|
||||
# --- Outpost Assignment ----------------------------------------------------
|
||||
- model: authentik_outposts.outpost
|
||||
state: present
|
||||
identifiers:
|
||||
name: "authentik Embedded Outpost"
|
||||
attrs:
|
||||
providers:
|
||||
- !KeyOf provider_qdrant
|
||||
- !KeyOf provider_neo4j
|
||||
|
||||
@@ -106,6 +106,7 @@ services:
|
||||
AUTHENTIK_MINIO_CLIENT_SECRET: ${AUTHENTIK_MINIO_CLIENT_SECRET}
|
||||
AUTHENTIK_VAULT_CLIENT_SECRET: ${AUTHENTIK_VAULT_CLIENT_SECRET}
|
||||
AUTHENTIK_BOOTSTRAP_FILE: /blueprints/ai-tax-agent-bootstrap.yaml
|
||||
AUTHENTIK_HOST_BROWSER: https://auth.${DOMAIN}:8444
|
||||
depends_on:
|
||||
- apa-authentik-db
|
||||
- apa-authentik-redis
|
||||
@@ -174,10 +175,11 @@ services:
|
||||
- frontend
|
||||
volumes:
|
||||
- vault_data:/vault/data
|
||||
- ./vault/config:/vault/config
|
||||
environment:
|
||||
VAULT_DEV_ROOT_TOKEN_ID: ${VAULT_DEV_ROOT_TOKEN_ID}
|
||||
VAULT_DEV_LISTEN_ADDRESS: 0.0.0.0:8200
|
||||
command: vault server -dev -dev-listen-address=0.0.0.0:8200
|
||||
VAULT_ADDR: "http://127.0.0.1:8200"
|
||||
VAULT_API_ADDR: "http://127.0.0.1:8200"
|
||||
command: vault server -config=/vault/config/vault.hcl
|
||||
cap_add:
|
||||
- IPC_LOCK
|
||||
extra_hosts:
|
||||
|
||||
@@ -63,11 +63,11 @@ services:
|
||||
GF_AUTH_GENERIC_OAUTH_ENABLED: true
|
||||
GF_AUTH_GENERIC_OAUTH_NAME: Authentik
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GRAFANA_OAUTH_CLIENT_ID}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GRAFANA_OAUTH_CLIENT_SECRET}
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${AUTHENTIK_GRAFANA_CLIENT_SECRET}
|
||||
GF_AUTH_GENERIC_OAUTH_SCOPES: openid profile email groups
|
||||
GF_AUTH_GENERIC_OAUTH_AUTH_URL: https://auth.${DOMAIN}/application/o/authorize/
|
||||
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: https://auth.${DOMAIN}/application/o/token/
|
||||
GF_AUTH_GENERIC_OAUTH_API_URL: https://auth.${DOMAIN}/application/o/userinfo/
|
||||
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: http://apa-authentik-server:9000/application/o/token/
|
||||
GF_AUTH_GENERIC_OAUTH_API_URL: http://apa-authentik-server:9000/application/o/userinfo/
|
||||
GF_AUTH_GENERIC_OAUTH_AUTO_LOGIN: false
|
||||
GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP: true
|
||||
GF_AUTH_GENERIC_OAUTH_ROLE_ATTRIBUTE_PATH: role
|
||||
|
||||
13
infra/base/vault/config/vault.hcl
Normal file
13
infra/base/vault/config/vault.hcl
Normal file
@@ -0,0 +1,13 @@
|
||||
storage "raft" {
|
||||
path = "/vault/data"
|
||||
node_id = "node1"
|
||||
}
|
||||
|
||||
listener "tcp" {
|
||||
address = "0.0.0.0:8200"
|
||||
tls_disable = "true"
|
||||
}
|
||||
|
||||
api_addr = "http://127.0.0.1:8200"
|
||||
cluster_addr = "https://127.0.0.1:8201"
|
||||
ui = true
|
||||
@@ -3,6 +3,7 @@
|
||||
This folder is for the self-contained local stack (self-signed TLS) and Traefik assets. Remote environments use the shared compose files in `infra/base` together with `infra/scripts/deploy.sh`.
|
||||
|
||||
## Local development (self-signed TLS)
|
||||
|
||||
- Copy envs: `cp infra/compose/env.example infra/compose/.env` then set passwords/secrets and the dev domain (defaults to `local.lan`).
|
||||
- Host aliases: add the domain to `/etc/hosts` (e.g. `127.0.0.1 auth.local.lan api.local.lan grafana.local.lan vault.local.lan minio.local.lan`).
|
||||
- Networks: `./infra/scripts/setup-networks.sh` (creates `apa-frontend` and `apa-backend` used everywhere).
|
||||
@@ -11,12 +12,35 @@ This folder is for the self-contained local stack (self-signed TLS) and Traefik
|
||||
- TLS: Traefik mounts `infra/compose/traefik/certs/local.{crt,key}`. Regenerate if needed with `openssl req -x509 -newkey rsa:2048 -nodes -keyout infra/compose/traefik/certs/local.key -out infra/compose/traefik/certs/local.crt -days 365 -subj "/CN=*.local.lan"`.
|
||||
|
||||
## Cloud / remote (Let’s Encrypt)
|
||||
- Config lives in `infra/base` with env files in `infra/environments/{development,production}/.env`.
|
||||
- Create the same docker networks on the host (`./infra/scripts/setup-networks.sh`) so Traefik and services share `apa-frontend` / `apa-backend`.
|
||||
- Deploy on the server: `./infra/scripts/deploy.sh <environment> all` (or `infrastructure`, `monitoring`, `services`).
|
||||
- Certificates: Traefik uses DNS-01 via GoDaddy from the provider env in `infra/base/traefik/config` (make sure `DOMAIN`, ACME email, and provider creds are set in the env file).
|
||||
|
||||
- **Structure**:
|
||||
|
||||
- `infra/environments/production/` contains the unified `compose.yaml` and `compose.override.yaml` (for port binding).
|
||||
- `infra/base/` contains the shared service definitions.
|
||||
- `infra/scripts/deploy.sh` is the deployment automation script.
|
||||
|
||||
- **Configuration**:
|
||||
|
||||
- Create `infra/environments/production/.env` based on `.env.example`.
|
||||
- Ensure `DOMAIN` is set (e.g., `app.harkon.co.uk`).
|
||||
- Ensure `AUTHENTIK_BOOTSTRAP_EMAIL` matches the blueprint (e.g., `admin@app.harkon.co.uk`).
|
||||
- Ensure `AUTHENTIK_BOOTSTRAP_FILE` is set to `./authentik/bootstrap-prod.yaml`.
|
||||
|
||||
- **Deployment**:
|
||||
|
||||
1. SSH into the server.
|
||||
2. Navigate to the project directory.
|
||||
3. Run: `./infra/scripts/deploy.sh production all`
|
||||
- This deploys infrastructure, monitoring, and application services.
|
||||
- It automatically uses the production compose files and environment variables.
|
||||
|
||||
- **Certificates**:
|
||||
- Traefik uses DNS-01 via GoDaddy (configured in `infra/environments/production/.env`).
|
||||
- Ensure `GODADDY_API_KEY` and `GODADDY_API_SECRET` are set.
|
||||
- Certificates are stored in `infra/base/certs/` (mapped to `/var/traefik/certs` in container).
|
||||
|
||||
## Files of note
|
||||
|
||||
- `docker-compose.local.yml` – full local stack.
|
||||
- `traefik/traefik.local.yml` and `traefik/traefik-dynamic.local.yml` – static/dynamic Traefik config for local.
|
||||
- `traefik/certs/` – self-signed certs used by the local proxy.
|
||||
|
||||
@@ -36,7 +36,8 @@ AUTHENTIK_BOOTSTRAP_TOKEN=
|
||||
|
||||
# Monitoring (CHANGE THIS!)
|
||||
GRAFANA_PASSWORD=CHANGE_ME_GRAFANA_PASSWORD
|
||||
GRAFANA_OAUTH_CLIENT_ID=grafana
|
||||
GRAFANA_OAUTH_CLIENT_ID=grafana-prod
|
||||
# MUST MATCH AUTHENTIK_GRAFANA_CLIENT_SECRET below
|
||||
GRAFANA_OAUTH_CLIENT_SECRET=CHANGE_ME_GRAFANA_OAUTH_SECRET
|
||||
|
||||
# OAuth Client Secrets for Authentik Providers (CHANGE THESE!)
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
services:
|
||||
apa-traefik:
|
||||
ports:
|
||||
- "80:80"
|
||||
- "443:443"
|
||||
- "8080:8080" # Dashboard (protected by middleware)
|
||||
# Reverting to base ports (8090/8444) to avoid conflict with existing Traefik
|
||||
# ports:
|
||||
# - "80:80"
|
||||
# - "443:443"
|
||||
# - "8080:8080"
|
||||
|
||||
@@ -210,6 +210,18 @@ deploy_all() {
|
||||
if [ "$ENVIRONMENT" = "local" ]; then
|
||||
log_info "Deploying unified stack for local environment..."
|
||||
compose_cmd "all" up -d "$@"
|
||||
elif [ "$ENVIRONMENT" = "production" ]; then
|
||||
log_info "Deploying unified stack for production environment..."
|
||||
local cmd="docker compose"
|
||||
cmd="$cmd -f $BASE_DIR/infrastructure.yaml"
|
||||
cmd="$cmd -f $BASE_DIR/services.yaml"
|
||||
cmd="$cmd -f $BASE_DIR/monitoring.yaml"
|
||||
|
||||
if [ -f "$INFRA_DIR/environments/$ENVIRONMENT/compose.override.yaml" ]; then
|
||||
cmd="$cmd -f $INFRA_DIR/environments/$ENVIRONMENT/compose.override.yaml"
|
||||
fi
|
||||
|
||||
$cmd --env-file "$ENV_FILE" --project-name "ai-tax-agent-$ENVIRONMENT" up -d "$@"
|
||||
elif [ -f "$unified_compose" ]; then
|
||||
log_info "Deploying unified stack for $ENVIRONMENT environment..."
|
||||
docker compose -f "$unified_compose" --env-file "$ENV_FILE" --project-name "ai-tax-agent-$ENVIRONMENT" up -d "$@"
|
||||
@@ -225,6 +237,27 @@ deploy_all() {
|
||||
|
||||
log_success "All stacks deployed successfully!"
|
||||
echo ""
|
||||
|
||||
# Post-deployment setup for Production
|
||||
if [ "$ENVIRONMENT" = "production" ]; then
|
||||
log_info "Running post-deployment setup..."
|
||||
|
||||
# Vault Setup
|
||||
if [ -f "$INFRA_DIR/scripts/init-vault.sh" ]; then
|
||||
log_info "Initializing/Unsealing Vault..."
|
||||
chmod +x "$INFRA_DIR/scripts/init-vault.sh"
|
||||
# Wait for Vault to be ready
|
||||
sleep 10
|
||||
"$INFRA_DIR/scripts/init-vault.sh"
|
||||
fi
|
||||
|
||||
if [ -f "$INFRA_DIR/scripts/setup-vault.sh" ]; then
|
||||
log_info "Configuring Vault OIDC..."
|
||||
chmod +x "$INFRA_DIR/scripts/setup-vault.sh"
|
||||
"$INFRA_DIR/scripts/setup-vault.sh"
|
||||
fi
|
||||
fi
|
||||
|
||||
log_info "Access your services:"
|
||||
echo " - Grafana: https://grafana.$DOMAIN"
|
||||
echo " - Prometheus: https://prometheus.$DOMAIN"
|
||||
|
||||
45
infra/scripts/init-vault.sh
Normal file
45
infra/scripts/init-vault.sh
Normal file
@@ -0,0 +1,45 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Load environment variables
|
||||
source infra/environments/production/.env
|
||||
|
||||
VAULT_ADDR="http://127.0.0.1:8200"
|
||||
CONTAINER_NAME="apa-vault"
|
||||
KEYS_FILE="infra/environments/production/.vault-keys"
|
||||
|
||||
echo "Checking Vault status..."
|
||||
|
||||
# Helper function to run vault commands inside docker
|
||||
vault_cmd() {
|
||||
docker exec -i -e VAULT_ADDR=$VAULT_ADDR $CONTAINER_NAME vault "$@"
|
||||
}
|
||||
|
||||
# Check if Vault is initialized
|
||||
if vault_cmd status -format=json | grep -q '"initialized": true'; then
|
||||
echo "Vault is already initialized."
|
||||
else
|
||||
echo "Vault is NOT initialized. Initializing..."
|
||||
INIT_OUTPUT=$(vault_cmd operator init -key-shares=1 -key-threshold=1 -format=json)
|
||||
|
||||
echo "$INIT_OUTPUT" > "$KEYS_FILE"
|
||||
chmod 600 "$KEYS_FILE"
|
||||
|
||||
echo "Vault initialized! Keys saved to $KEYS_FILE"
|
||||
echo "WARNING: BACK UP THIS FILE SECURELY!"
|
||||
fi
|
||||
|
||||
# Read keys
|
||||
# Extract first key from the array (assuming 1 key share)
|
||||
UNSEAL_KEY=$(grep -A 1 '"unseal_keys_b64":' "$KEYS_FILE" | tail -n 1 | cut -d'"' -f2)
|
||||
ROOT_TOKEN=$(grep '"root_token":' "$KEYS_FILE" | cut -d'"' -f4)
|
||||
|
||||
# Unseal
|
||||
echo "Unsealing Vault..."
|
||||
vault_cmd operator unseal "$UNSEAL_KEY"
|
||||
|
||||
echo "Vault is Unsealed!"
|
||||
echo "Root Token: $ROOT_TOKEN"
|
||||
|
||||
# Export Root Token for setup script
|
||||
export VAULT_TOKEN=$ROOT_TOKEN
|
||||
84
infra/scripts/setup-vault.sh
Normal file
84
infra/scripts/setup-vault.sh
Normal file
@@ -0,0 +1,84 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Load environment variables
|
||||
source infra/environments/production/.env
|
||||
|
||||
# Vault Configuration
|
||||
VAULT_ADDR="http://127.0.0.1:8200"
|
||||
KEYS_FILE="infra/environments/production/.vault-keys"
|
||||
|
||||
if [ ! -f "$KEYS_FILE" ]; then
|
||||
echo "Error: Keys file not found at $KEYS_FILE. Run init-vault.sh first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
VAULT_TOKEN=$(grep '"root_token":' "$KEYS_FILE" | cut -d'"' -f4)
|
||||
CONTAINER_NAME="apa-vault"
|
||||
|
||||
echo "Configuring Vault..."
|
||||
|
||||
# Helper function to run vault commands inside docker
|
||||
vault_cmd() {
|
||||
docker exec -i -e VAULT_ADDR=$VAULT_ADDR -e VAULT_TOKEN=$VAULT_TOKEN $CONTAINER_NAME vault "$@"
|
||||
}
|
||||
|
||||
# Enable OIDC auth method
|
||||
echo "Enabling OIDC auth method..."
|
||||
if ! vault_cmd auth list | grep -q "oidc/"; then
|
||||
vault_cmd auth enable oidc
|
||||
else
|
||||
echo "OIDC auth method already enabled."
|
||||
fi
|
||||
|
||||
# Configure OIDC
|
||||
echo "Configuring OIDC..."
|
||||
vault_cmd write auth/oidc/config \
|
||||
oidc_discovery_url="https://auth.${DOMAIN}/application/o/vault-prod/" \
|
||||
oidc_client_id="vault-prod" \
|
||||
oidc_client_secret="${AUTHENTIK_VAULT_CLIENT_SECRET}" \
|
||||
default_role="reader"
|
||||
|
||||
# Create Policies
|
||||
echo "Creating policies..."
|
||||
|
||||
# Admin Policy
|
||||
vault_cmd policy write admin - <<EOF
|
||||
path "*" {
|
||||
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
|
||||
}
|
||||
EOF
|
||||
|
||||
# Reader Policy
|
||||
vault_cmd policy write reader - <<EOF
|
||||
path "secret/*" {
|
||||
capabilities = ["read", "list"]
|
||||
}
|
||||
EOF
|
||||
|
||||
# Create Roles
|
||||
echo "Creating roles..."
|
||||
|
||||
# Admin Role
|
||||
vault_cmd write auth/oidc/role/admin \
|
||||
bound_audiences="vault-prod" \
|
||||
allowed_redirect_uris="https://vault.${DOMAIN}/ui/vault/auth/oidc/oidc/callback" \
|
||||
allowed_redirect_uris="https://vault.${DOMAIN}/oidc/callback" \
|
||||
user_claim="email" \
|
||||
policies="admin" \
|
||||
role_type="oidc" \
|
||||
groups_claim="groups" \
|
||||
oidc_scopes="openid,email,profile,groups"
|
||||
|
||||
# Reader Role
|
||||
vault_cmd write auth/oidc/role/reader \
|
||||
bound_audiences="vault-prod" \
|
||||
allowed_redirect_uris="https://vault.${DOMAIN}/ui/vault/auth/oidc/oidc/callback" \
|
||||
allowed_redirect_uris="https://vault.${DOMAIN}/oidc/callback" \
|
||||
user_claim="email" \
|
||||
policies="reader" \
|
||||
role_type="oidc" \
|
||||
groups_claim="groups" \
|
||||
oidc_scopes="openid,email,profile,groups"
|
||||
|
||||
echo "Vault configuration complete!"
|
||||
Reference in New Issue
Block a user