feat: working infra with sso
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
Some checks failed
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
This commit is contained in:
23
README.md
23
README.md
@@ -156,8 +156,8 @@ The deployment automation handles:
|
|||||||
### Project Structure
|
### Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
ai-tax-agent-2/
|
ai-tax-agent/
|
||||||
├── libs/ # Shared libraries
|
├── libs/ # Shared libraries
|
||||||
│ ├── config.py # Configuration and factories
|
│ ├── config.py # Configuration and factories
|
||||||
│ ├── security.py # Authentication and encryption
|
│ ├── security.py # Authentication and encryption
|
||||||
│ ├── observability.py # Tracing, metrics, logging
|
│ ├── observability.py # Tracing, metrics, logging
|
||||||
@@ -356,20 +356,17 @@ make clean # Clean up volumes and networks
|
|||||||
|
|
||||||
### Production Deployment
|
### Production Deployment
|
||||||
|
|
||||||
|
For detailed instructions, see [infra/compose/README.md](infra/compose/README.md).
|
||||||
|
|
||||||
|
The system uses a unified deployment script for production environments:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Using Docker Swarm
|
# Deploy to production (Infrastructure + Services + Monitoring)
|
||||||
make deploy-swarm
|
./infra/scripts/deploy.sh production all
|
||||||
|
|
||||||
# Using Kubernetes
|
|
||||||
make deploy-k8s
|
|
||||||
|
|
||||||
# Using Terraform (AWS/Azure/GCP)
|
|
||||||
cd infra/terraform
|
|
||||||
terraform init
|
|
||||||
terraform plan
|
|
||||||
terraform apply
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Ensure you have configured `infra/environments/production/.env` with the correct secrets and domain settings before deploying.
|
||||||
|
|
||||||
### Environment Configuration
|
### Environment Configuration
|
||||||
|
|
||||||
Key environment variables:
|
Key environment variables:
|
||||||
|
|||||||
@@ -1,85 +0,0 @@
|
|||||||
# Authentik Configuration - Grafana SSO Integration
|
|
||||||
# Generated: 2025-09-20 07:25:00
|
|
||||||
# This file contains the Authentik configuration for Grafana OAuth2/OIDC integration
|
|
||||||
# Apply this blueprint to automate the setup of Grafana SSO with Authentik
|
|
||||||
|
|
||||||
version: 1
|
|
||||||
|
|
||||||
metadata:
|
|
||||||
name: AI Tax Agent Grafana SSO Integration
|
|
||||||
labels:
|
|
||||||
blueprints.goauthentik.io/generated: "true"
|
|
||||||
|
|
||||||
entries:
|
|
||||||
# Grafana OAuth2 Provider
|
|
||||||
- attrs:
|
|
||||||
authorization_flow: !Find [authentik_flows.flow, [slug, default-provider-authorization-implicit-consent]]
|
|
||||||
invalidation_flow: !Find [authentik_flows.flow, [slug, default-provider-invalidation-flow]]
|
|
||||||
name: grafana
|
|
||||||
client_type: confidential
|
|
||||||
client_id: grafana
|
|
||||||
client_secret: ${AUTHENTIK_GRAFANA_CLIENT_SECRET:-changeme-grafana-secret}
|
|
||||||
redirect_uris:
|
|
||||||
- https://grafana.${DOMAIN:-local.lan}/login/generic_oauth
|
|
||||||
sub_mode: hashed_user_id
|
|
||||||
include_claims_in_id_token: true
|
|
||||||
issuer_mode: per_provider
|
|
||||||
signing_key: !Find [authentik_crypto.certificatekeypair, [name, authentik Self-signed Certificate]]
|
|
||||||
property_mappings:
|
|
||||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, openid]]
|
|
||||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, email]]
|
|
||||||
- !Find [authentik_providers_oauth2.scopemapping, [scope_name, profile]]
|
|
||||||
- !KeyOf grafana-groups-mapping
|
|
||||||
conditions: []
|
|
||||||
identifiers:
|
|
||||||
name: grafana
|
|
||||||
model: authentik_providers_oauth2.oauth2provider
|
|
||||||
permissions: []
|
|
||||||
state: present
|
|
||||||
|
|
||||||
# Custom Groups Mapping for Grafana
|
|
||||||
- attrs:
|
|
||||||
name: Grafana Groups Mapping
|
|
||||||
description: Maps Authentik groups to Grafana roles
|
|
||||||
scope_name: groups
|
|
||||||
expression: |
|
|
||||||
# Map Authentik groups to Grafana roles
|
|
||||||
groups = []
|
|
||||||
user_groups = [group.name for group in request.user.ak_groups.all()]
|
|
||||||
|
|
||||||
# Admin role mapping
|
|
||||||
if "authentik Admins" in user_groups or "Administrators" in user_groups:
|
|
||||||
groups.append("Admin")
|
|
||||||
|
|
||||||
# Editor role mapping
|
|
||||||
if "Tax Reviewers" in user_groups or "Accountants" in user_groups:
|
|
||||||
groups.append("Editor")
|
|
||||||
|
|
||||||
# Viewer role mapping (default for all authenticated users)
|
|
||||||
groups.append("Viewer")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"groups": groups,
|
|
||||||
"role": groups[0] if groups else "Viewer" # Primary role
|
|
||||||
}
|
|
||||||
conditions: []
|
|
||||||
identifiers:
|
|
||||||
name: Grafana Groups Mapping
|
|
||||||
model: authentik_providers_oauth2.scopemapping
|
|
||||||
permissions: []
|
|
||||||
state: present
|
|
||||||
|
|
||||||
# Grafana Application
|
|
||||||
- attrs:
|
|
||||||
name: Grafana
|
|
||||||
slug: grafana
|
|
||||||
provider: !KeyOf grafana
|
|
||||||
policy_engine_mode: any
|
|
||||||
meta_description: Grafana monitoring and observability platform
|
|
||||||
meta_publisher: Grafana Labs
|
|
||||||
conditions: []
|
|
||||||
identifiers:
|
|
||||||
slug: grafana
|
|
||||||
model: authentik_core.application
|
|
||||||
permissions: []
|
|
||||||
state: present
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
# Simple Authentik Bootstrap Configuration
|
|
||||||
# This file configures the basic Authentik setup for AI Tax Agent
|
|
||||||
|
|
||||||
version: 1
|
|
||||||
|
|
||||||
metadata:
|
|
||||||
name: AI Tax Agent Simple Bootstrap
|
|
||||||
|
|
||||||
entries:
|
|
||||||
# Create admin user
|
|
||||||
- model: authentik_core.user
|
|
||||||
identifiers:
|
|
||||||
username: admin
|
|
||||||
attrs:
|
|
||||||
name: "System Administrator"
|
|
||||||
email: admin@local.lan
|
|
||||||
is_active: true
|
|
||||||
is_staff: true
|
|
||||||
is_superuser: true
|
|
||||||
|
|
||||||
# Create user groups
|
|
||||||
- model: authentik_core.group
|
|
||||||
identifiers:
|
|
||||||
name: "Administrators"
|
|
||||||
attrs:
|
|
||||||
is_superuser: true
|
|
||||||
|
|
||||||
- model: authentik_core.group
|
|
||||||
identifiers:
|
|
||||||
name: "Tax Reviewers"
|
|
||||||
attrs:
|
|
||||||
is_superuser: false
|
|
||||||
|
|
||||||
- model: authentik_core.group
|
|
||||||
identifiers:
|
|
||||||
name: "Accountants"
|
|
||||||
attrs:
|
|
||||||
is_superuser: false
|
|
||||||
|
|
||||||
- model: authentik_core.group
|
|
||||||
identifiers:
|
|
||||||
name: "Clients"
|
|
||||||
attrs:
|
|
||||||
is_superuser: false
|
|
||||||
|
|
||||||
# Create OIDC Provider for API services
|
|
||||||
- model: authentik_providers_oauth2.oauth2provider
|
|
||||||
identifiers:
|
|
||||||
name: "AI Tax Agent API"
|
|
||||||
attrs:
|
|
||||||
client_id: "ai-tax-agent-api"
|
|
||||||
client_secret: !Env [AUTHENTIK_API_CLIENT_SECRET, "changeme-api-secret"]
|
|
||||||
authorization_grant_type: "authorization-code"
|
|
||||||
client_type: "confidential"
|
|
||||||
redirect_uris: "https://api.local/auth/callback\nhttps://review.local/auth/callback"
|
|
||||||
sub_mode: "hashed_user_id"
|
|
||||||
include_claims_in_id_token: true
|
|
||||||
issuer_mode: "per_provider"
|
|
||||||
authorization_flow:
|
|
||||||
!Find [authentik_flows.flow, [slug, "default-authentication-flow"]]
|
|
||||||
invalidation_flow:
|
|
||||||
!Find [authentik_flows.flow, [slug, "default-invalidation-flow"]]
|
|
||||||
|
|
||||||
# Create OIDC Provider for Grafana
|
|
||||||
- model: authentik_providers_oauth2.oauth2provider
|
|
||||||
identifiers:
|
|
||||||
name: "Grafana"
|
|
||||||
attrs:
|
|
||||||
client_id: "grafana"
|
|
||||||
client_secret:
|
|
||||||
!Env [AUTHENTIK_GRAFANA_CLIENT_SECRET, "changeme-grafana-secret"]
|
|
||||||
authorization_grant_type: "authorization-code"
|
|
||||||
client_type: "confidential"
|
|
||||||
redirect_uris: "https://grafana.local/login/generic_oauth"
|
|
||||||
sub_mode: "hashed_user_id"
|
|
||||||
include_claims_in_id_token: true
|
|
||||||
issuer_mode: "per_provider"
|
|
||||||
authorization_flow:
|
|
||||||
!Find [authentik_flows.flow, [slug, "default-authentication-flow"]]
|
|
||||||
invalidation_flow:
|
|
||||||
!Find [authentik_flows.flow, [slug, "default-invalidation-flow"]]
|
|
||||||
|
|
||||||
# Create Applications
|
|
||||||
- model: authentik_core.application
|
|
||||||
identifiers:
|
|
||||||
name: "AI Tax Agent API"
|
|
||||||
slug: "ai-tax-agent-api"
|
|
||||||
attrs:
|
|
||||||
provider:
|
|
||||||
!Find [
|
|
||||||
authentik_providers_oauth2.oauth2provider,
|
|
||||||
[name, "AI Tax Agent API"],
|
|
||||||
]
|
|
||||||
meta_launch_url: "https://api.local"
|
|
||||||
meta_description: "AI Tax Agent API Services"
|
|
||||||
meta_publisher: "AI Tax Agent"
|
|
||||||
policy_engine_mode: "any"
|
|
||||||
|
|
||||||
- model: authentik_core.application
|
|
||||||
identifiers:
|
|
||||||
name: "Grafana"
|
|
||||||
slug: "grafana"
|
|
||||||
attrs:
|
|
||||||
provider:
|
|
||||||
!Find [authentik_providers_oauth2.oauth2provider, [name, "Grafana"]]
|
|
||||||
meta_launch_url: "https://grafana.local"
|
|
||||||
meta_description: "Monitoring and Observability Dashboard"
|
|
||||||
meta_publisher: "AI Tax Agent"
|
|
||||||
policy_engine_mode: "any"
|
|
||||||
@@ -167,6 +167,7 @@ entries:
|
|||||||
- !KeyOf scope_email
|
- !KeyOf scope_email
|
||||||
- !KeyOf scope_groups
|
- !KeyOf scope_groups
|
||||||
- !KeyOf scope_offline
|
- !KeyOf scope_offline
|
||||||
|
- !KeyOf scope_minio_policy
|
||||||
authorization_flow: !KeyOf default_authz_flow
|
authorization_flow: !KeyOf default_authz_flow
|
||||||
invalidation_flow: !KeyOf default_inval_flow
|
invalidation_flow: !KeyOf default_inval_flow
|
||||||
|
|
||||||
@@ -258,6 +259,31 @@ entries:
|
|||||||
# Default to Viewer role
|
# Default to Viewer role
|
||||||
return "Viewer"
|
return "Viewer"
|
||||||
|
|
||||||
|
# Custom Scope Mapping for MinIO
|
||||||
|
- id: scope_minio_policy
|
||||||
|
model: authentik_providers_oauth2.scopemapping
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
name: "MinIO Policy Mapping"
|
||||||
|
attrs:
|
||||||
|
name: "MinIO Policy Mapping"
|
||||||
|
description: "Maps Authentik groups to MinIO policies"
|
||||||
|
scope_name: "minio"
|
||||||
|
expression: |
|
||||||
|
# Map Authentik groups to MinIO policies
|
||||||
|
user_groups = [group.name for group in request.user.ak_groups.all()]
|
||||||
|
|
||||||
|
# Admin policy mapping
|
||||||
|
if "Administrators" in user_groups:
|
||||||
|
return {"policy": ["consoleAdmin"]}
|
||||||
|
|
||||||
|
# Default to readwrite for other known groups
|
||||||
|
if "Tax Reviewers" in user_groups or "Accountants" in user_groups:
|
||||||
|
return {"policy": ["readwrite"]}
|
||||||
|
|
||||||
|
# Default fallback
|
||||||
|
return {"policy": ["readonly"]}
|
||||||
|
|
||||||
- model: authentik_providers_oauth2.oauth2provider
|
- model: authentik_providers_oauth2.oauth2provider
|
||||||
state: present
|
state: present
|
||||||
identifiers:
|
identifiers:
|
||||||
@@ -340,3 +366,78 @@ entries:
|
|||||||
target: !Find [authentik_core.application, [slug, "grafana-prod"]]
|
target: !Find [authentik_core.application, [slug, "grafana-prod"]]
|
||||||
attrs:
|
attrs:
|
||||||
order: 0
|
order: 0
|
||||||
|
|
||||||
|
# --- Qdrant (Production) ---------------------------------------------------
|
||||||
|
- id: provider_qdrant
|
||||||
|
model: authentik_providers_proxy.proxyprovider
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
name: "Qdrant (Production)"
|
||||||
|
attrs:
|
||||||
|
external_host: "https://qdrant.app.harkon.co.uk:8444"
|
||||||
|
authorization_flow: !KeyOf default_authz_flow
|
||||||
|
invalidation_flow: !KeyOf default_inval_flow
|
||||||
|
mode: "forward_single"
|
||||||
|
|
||||||
|
- model: authentik_core.application
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
slug: "qdrant-prod"
|
||||||
|
attrs:
|
||||||
|
name: "Qdrant (Production)"
|
||||||
|
provider: !KeyOf provider_qdrant
|
||||||
|
meta_launch_url: "https://qdrant.app.harkon.co.uk:8444/dashboard"
|
||||||
|
meta_description: "Vector Database Dashboard"
|
||||||
|
meta_publisher: "AI Tax Agent"
|
||||||
|
policy_engine_mode: "any"
|
||||||
|
|
||||||
|
# --- Neo4j (Production) ----------------------------------------------------
|
||||||
|
- id: provider_neo4j
|
||||||
|
model: authentik_providers_proxy.proxyprovider
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
name: "Neo4j (Production)"
|
||||||
|
attrs:
|
||||||
|
external_host: "https://neo4j.app.harkon.co.uk:8444"
|
||||||
|
authorization_flow: !KeyOf default_authz_flow
|
||||||
|
invalidation_flow: !KeyOf default_inval_flow
|
||||||
|
mode: "forward_single"
|
||||||
|
|
||||||
|
- model: authentik_core.application
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
slug: "neo4j-prod"
|
||||||
|
attrs:
|
||||||
|
name: "Neo4j (Production)"
|
||||||
|
provider: !KeyOf provider_neo4j
|
||||||
|
meta_launch_url: "https://neo4j.app.harkon.co.uk:8444"
|
||||||
|
meta_description: "Knowledge Graph Browser"
|
||||||
|
meta_publisher: "AI Tax Agent"
|
||||||
|
policy_engine_mode: "any"
|
||||||
|
|
||||||
|
# --- Policy Bindings for New Apps ------------------------------------------
|
||||||
|
- model: authentik_policies.policybinding
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
policy: !KeyOf policy_always_allow
|
||||||
|
target: !Find [authentik_core.application, [slug, "qdrant-prod"]]
|
||||||
|
attrs:
|
||||||
|
order: 0
|
||||||
|
|
||||||
|
- model: authentik_policies.policybinding
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
policy: !KeyOf policy_always_allow
|
||||||
|
target: !Find [authentik_core.application, [slug, "neo4j-prod"]]
|
||||||
|
attrs:
|
||||||
|
order: 0
|
||||||
|
|
||||||
|
# --- Outpost Assignment ----------------------------------------------------
|
||||||
|
- model: authentik_outposts.outpost
|
||||||
|
state: present
|
||||||
|
identifiers:
|
||||||
|
name: "authentik Embedded Outpost"
|
||||||
|
attrs:
|
||||||
|
providers:
|
||||||
|
- !KeyOf provider_qdrant
|
||||||
|
- !KeyOf provider_neo4j
|
||||||
|
|||||||
@@ -106,6 +106,7 @@ services:
|
|||||||
AUTHENTIK_MINIO_CLIENT_SECRET: ${AUTHENTIK_MINIO_CLIENT_SECRET}
|
AUTHENTIK_MINIO_CLIENT_SECRET: ${AUTHENTIK_MINIO_CLIENT_SECRET}
|
||||||
AUTHENTIK_VAULT_CLIENT_SECRET: ${AUTHENTIK_VAULT_CLIENT_SECRET}
|
AUTHENTIK_VAULT_CLIENT_SECRET: ${AUTHENTIK_VAULT_CLIENT_SECRET}
|
||||||
AUTHENTIK_BOOTSTRAP_FILE: /blueprints/ai-tax-agent-bootstrap.yaml
|
AUTHENTIK_BOOTSTRAP_FILE: /blueprints/ai-tax-agent-bootstrap.yaml
|
||||||
|
AUTHENTIK_HOST_BROWSER: https://auth.${DOMAIN}:8444
|
||||||
depends_on:
|
depends_on:
|
||||||
- apa-authentik-db
|
- apa-authentik-db
|
||||||
- apa-authentik-redis
|
- apa-authentik-redis
|
||||||
@@ -174,10 +175,11 @@ services:
|
|||||||
- frontend
|
- frontend
|
||||||
volumes:
|
volumes:
|
||||||
- vault_data:/vault/data
|
- vault_data:/vault/data
|
||||||
|
- ./vault/config:/vault/config
|
||||||
environment:
|
environment:
|
||||||
VAULT_DEV_ROOT_TOKEN_ID: ${VAULT_DEV_ROOT_TOKEN_ID}
|
VAULT_ADDR: "http://127.0.0.1:8200"
|
||||||
VAULT_DEV_LISTEN_ADDRESS: 0.0.0.0:8200
|
VAULT_API_ADDR: "http://127.0.0.1:8200"
|
||||||
command: vault server -dev -dev-listen-address=0.0.0.0:8200
|
command: vault server -config=/vault/config/vault.hcl
|
||||||
cap_add:
|
cap_add:
|
||||||
- IPC_LOCK
|
- IPC_LOCK
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
|
|||||||
@@ -63,11 +63,11 @@ services:
|
|||||||
GF_AUTH_GENERIC_OAUTH_ENABLED: true
|
GF_AUTH_GENERIC_OAUTH_ENABLED: true
|
||||||
GF_AUTH_GENERIC_OAUTH_NAME: Authentik
|
GF_AUTH_GENERIC_OAUTH_NAME: Authentik
|
||||||
GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GRAFANA_OAUTH_CLIENT_ID}
|
GF_AUTH_GENERIC_OAUTH_CLIENT_ID: ${GRAFANA_OAUTH_CLIENT_ID}
|
||||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GRAFANA_OAUTH_CLIENT_SECRET}
|
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${AUTHENTIK_GRAFANA_CLIENT_SECRET}
|
||||||
GF_AUTH_GENERIC_OAUTH_SCOPES: openid profile email groups
|
GF_AUTH_GENERIC_OAUTH_SCOPES: openid profile email groups
|
||||||
GF_AUTH_GENERIC_OAUTH_AUTH_URL: https://auth.${DOMAIN}/application/o/authorize/
|
GF_AUTH_GENERIC_OAUTH_AUTH_URL: https://auth.${DOMAIN}/application/o/authorize/
|
||||||
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: https://auth.${DOMAIN}/application/o/token/
|
GF_AUTH_GENERIC_OAUTH_TOKEN_URL: http://apa-authentik-server:9000/application/o/token/
|
||||||
GF_AUTH_GENERIC_OAUTH_API_URL: https://auth.${DOMAIN}/application/o/userinfo/
|
GF_AUTH_GENERIC_OAUTH_API_URL: http://apa-authentik-server:9000/application/o/userinfo/
|
||||||
GF_AUTH_GENERIC_OAUTH_AUTO_LOGIN: false
|
GF_AUTH_GENERIC_OAUTH_AUTO_LOGIN: false
|
||||||
GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP: true
|
GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP: true
|
||||||
GF_AUTH_GENERIC_OAUTH_ROLE_ATTRIBUTE_PATH: role
|
GF_AUTH_GENERIC_OAUTH_ROLE_ATTRIBUTE_PATH: role
|
||||||
|
|||||||
13
infra/base/vault/config/vault.hcl
Normal file
13
infra/base/vault/config/vault.hcl
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
storage "raft" {
|
||||||
|
path = "/vault/data"
|
||||||
|
node_id = "node1"
|
||||||
|
}
|
||||||
|
|
||||||
|
listener "tcp" {
|
||||||
|
address = "0.0.0.0:8200"
|
||||||
|
tls_disable = "true"
|
||||||
|
}
|
||||||
|
|
||||||
|
api_addr = "http://127.0.0.1:8200"
|
||||||
|
cluster_addr = "https://127.0.0.1:8201"
|
||||||
|
ui = true
|
||||||
@@ -3,6 +3,7 @@
|
|||||||
This folder is for the self-contained local stack (self-signed TLS) and Traefik assets. Remote environments use the shared compose files in `infra/base` together with `infra/scripts/deploy.sh`.
|
This folder is for the self-contained local stack (self-signed TLS) and Traefik assets. Remote environments use the shared compose files in `infra/base` together with `infra/scripts/deploy.sh`.
|
||||||
|
|
||||||
## Local development (self-signed TLS)
|
## Local development (self-signed TLS)
|
||||||
|
|
||||||
- Copy envs: `cp infra/compose/env.example infra/compose/.env` then set passwords/secrets and the dev domain (defaults to `local.lan`).
|
- Copy envs: `cp infra/compose/env.example infra/compose/.env` then set passwords/secrets and the dev domain (defaults to `local.lan`).
|
||||||
- Host aliases: add the domain to `/etc/hosts` (e.g. `127.0.0.1 auth.local.lan api.local.lan grafana.local.lan vault.local.lan minio.local.lan`).
|
- Host aliases: add the domain to `/etc/hosts` (e.g. `127.0.0.1 auth.local.lan api.local.lan grafana.local.lan vault.local.lan minio.local.lan`).
|
||||||
- Networks: `./infra/scripts/setup-networks.sh` (creates `apa-frontend` and `apa-backend` used everywhere).
|
- Networks: `./infra/scripts/setup-networks.sh` (creates `apa-frontend` and `apa-backend` used everywhere).
|
||||||
@@ -11,12 +12,35 @@ This folder is for the self-contained local stack (self-signed TLS) and Traefik
|
|||||||
- TLS: Traefik mounts `infra/compose/traefik/certs/local.{crt,key}`. Regenerate if needed with `openssl req -x509 -newkey rsa:2048 -nodes -keyout infra/compose/traefik/certs/local.key -out infra/compose/traefik/certs/local.crt -days 365 -subj "/CN=*.local.lan"`.
|
- TLS: Traefik mounts `infra/compose/traefik/certs/local.{crt,key}`. Regenerate if needed with `openssl req -x509 -newkey rsa:2048 -nodes -keyout infra/compose/traefik/certs/local.key -out infra/compose/traefik/certs/local.crt -days 365 -subj "/CN=*.local.lan"`.
|
||||||
|
|
||||||
## Cloud / remote (Let’s Encrypt)
|
## Cloud / remote (Let’s Encrypt)
|
||||||
- Config lives in `infra/base` with env files in `infra/environments/{development,production}/.env`.
|
|
||||||
- Create the same docker networks on the host (`./infra/scripts/setup-networks.sh`) so Traefik and services share `apa-frontend` / `apa-backend`.
|
- **Structure**:
|
||||||
- Deploy on the server: `./infra/scripts/deploy.sh <environment> all` (or `infrastructure`, `monitoring`, `services`).
|
|
||||||
- Certificates: Traefik uses DNS-01 via GoDaddy from the provider env in `infra/base/traefik/config` (make sure `DOMAIN`, ACME email, and provider creds are set in the env file).
|
- `infra/environments/production/` contains the unified `compose.yaml` and `compose.override.yaml` (for port binding).
|
||||||
|
- `infra/base/` contains the shared service definitions.
|
||||||
|
- `infra/scripts/deploy.sh` is the deployment automation script.
|
||||||
|
|
||||||
|
- **Configuration**:
|
||||||
|
|
||||||
|
- Create `infra/environments/production/.env` based on `.env.example`.
|
||||||
|
- Ensure `DOMAIN` is set (e.g., `app.harkon.co.uk`).
|
||||||
|
- Ensure `AUTHENTIK_BOOTSTRAP_EMAIL` matches the blueprint (e.g., `admin@app.harkon.co.uk`).
|
||||||
|
- Ensure `AUTHENTIK_BOOTSTRAP_FILE` is set to `./authentik/bootstrap-prod.yaml`.
|
||||||
|
|
||||||
|
- **Deployment**:
|
||||||
|
|
||||||
|
1. SSH into the server.
|
||||||
|
2. Navigate to the project directory.
|
||||||
|
3. Run: `./infra/scripts/deploy.sh production all`
|
||||||
|
- This deploys infrastructure, monitoring, and application services.
|
||||||
|
- It automatically uses the production compose files and environment variables.
|
||||||
|
|
||||||
|
- **Certificates**:
|
||||||
|
- Traefik uses DNS-01 via GoDaddy (configured in `infra/environments/production/.env`).
|
||||||
|
- Ensure `GODADDY_API_KEY` and `GODADDY_API_SECRET` are set.
|
||||||
|
- Certificates are stored in `infra/base/certs/` (mapped to `/var/traefik/certs` in container).
|
||||||
|
|
||||||
## Files of note
|
## Files of note
|
||||||
|
|
||||||
- `docker-compose.local.yml` – full local stack.
|
- `docker-compose.local.yml` – full local stack.
|
||||||
- `traefik/traefik.local.yml` and `traefik/traefik-dynamic.local.yml` – static/dynamic Traefik config for local.
|
- `traefik/traefik.local.yml` and `traefik/traefik-dynamic.local.yml` – static/dynamic Traefik config for local.
|
||||||
- `traefik/certs/` – self-signed certs used by the local proxy.
|
- `traefik/certs/` – self-signed certs used by the local proxy.
|
||||||
|
|||||||
@@ -36,7 +36,8 @@ AUTHENTIK_BOOTSTRAP_TOKEN=
|
|||||||
|
|
||||||
# Monitoring (CHANGE THIS!)
|
# Monitoring (CHANGE THIS!)
|
||||||
GRAFANA_PASSWORD=CHANGE_ME_GRAFANA_PASSWORD
|
GRAFANA_PASSWORD=CHANGE_ME_GRAFANA_PASSWORD
|
||||||
GRAFANA_OAUTH_CLIENT_ID=grafana
|
GRAFANA_OAUTH_CLIENT_ID=grafana-prod
|
||||||
|
# MUST MATCH AUTHENTIK_GRAFANA_CLIENT_SECRET below
|
||||||
GRAFANA_OAUTH_CLIENT_SECRET=CHANGE_ME_GRAFANA_OAUTH_SECRET
|
GRAFANA_OAUTH_CLIENT_SECRET=CHANGE_ME_GRAFANA_OAUTH_SECRET
|
||||||
|
|
||||||
# OAuth Client Secrets for Authentik Providers (CHANGE THESE!)
|
# OAuth Client Secrets for Authentik Providers (CHANGE THESE!)
|
||||||
|
|||||||
@@ -4,7 +4,8 @@
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
apa-traefik:
|
apa-traefik:
|
||||||
ports:
|
# Reverting to base ports (8090/8444) to avoid conflict with existing Traefik
|
||||||
- "80:80"
|
# ports:
|
||||||
- "443:443"
|
# - "80:80"
|
||||||
- "8080:8080" # Dashboard (protected by middleware)
|
# - "443:443"
|
||||||
|
# - "8080:8080"
|
||||||
|
|||||||
@@ -210,6 +210,18 @@ deploy_all() {
|
|||||||
if [ "$ENVIRONMENT" = "local" ]; then
|
if [ "$ENVIRONMENT" = "local" ]; then
|
||||||
log_info "Deploying unified stack for local environment..."
|
log_info "Deploying unified stack for local environment..."
|
||||||
compose_cmd "all" up -d "$@"
|
compose_cmd "all" up -d "$@"
|
||||||
|
elif [ "$ENVIRONMENT" = "production" ]; then
|
||||||
|
log_info "Deploying unified stack for production environment..."
|
||||||
|
local cmd="docker compose"
|
||||||
|
cmd="$cmd -f $BASE_DIR/infrastructure.yaml"
|
||||||
|
cmd="$cmd -f $BASE_DIR/services.yaml"
|
||||||
|
cmd="$cmd -f $BASE_DIR/monitoring.yaml"
|
||||||
|
|
||||||
|
if [ -f "$INFRA_DIR/environments/$ENVIRONMENT/compose.override.yaml" ]; then
|
||||||
|
cmd="$cmd -f $INFRA_DIR/environments/$ENVIRONMENT/compose.override.yaml"
|
||||||
|
fi
|
||||||
|
|
||||||
|
$cmd --env-file "$ENV_FILE" --project-name "ai-tax-agent-$ENVIRONMENT" up -d "$@"
|
||||||
elif [ -f "$unified_compose" ]; then
|
elif [ -f "$unified_compose" ]; then
|
||||||
log_info "Deploying unified stack for $ENVIRONMENT environment..."
|
log_info "Deploying unified stack for $ENVIRONMENT environment..."
|
||||||
docker compose -f "$unified_compose" --env-file "$ENV_FILE" --project-name "ai-tax-agent-$ENVIRONMENT" up -d "$@"
|
docker compose -f "$unified_compose" --env-file "$ENV_FILE" --project-name "ai-tax-agent-$ENVIRONMENT" up -d "$@"
|
||||||
@@ -225,6 +237,27 @@ deploy_all() {
|
|||||||
|
|
||||||
log_success "All stacks deployed successfully!"
|
log_success "All stacks deployed successfully!"
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
|
# Post-deployment setup for Production
|
||||||
|
if [ "$ENVIRONMENT" = "production" ]; then
|
||||||
|
log_info "Running post-deployment setup..."
|
||||||
|
|
||||||
|
# Vault Setup
|
||||||
|
if [ -f "$INFRA_DIR/scripts/init-vault.sh" ]; then
|
||||||
|
log_info "Initializing/Unsealing Vault..."
|
||||||
|
chmod +x "$INFRA_DIR/scripts/init-vault.sh"
|
||||||
|
# Wait for Vault to be ready
|
||||||
|
sleep 10
|
||||||
|
"$INFRA_DIR/scripts/init-vault.sh"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -f "$INFRA_DIR/scripts/setup-vault.sh" ]; then
|
||||||
|
log_info "Configuring Vault OIDC..."
|
||||||
|
chmod +x "$INFRA_DIR/scripts/setup-vault.sh"
|
||||||
|
"$INFRA_DIR/scripts/setup-vault.sh"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
log_info "Access your services:"
|
log_info "Access your services:"
|
||||||
echo " - Grafana: https://grafana.$DOMAIN"
|
echo " - Grafana: https://grafana.$DOMAIN"
|
||||||
echo " - Prometheus: https://prometheus.$DOMAIN"
|
echo " - Prometheus: https://prometheus.$DOMAIN"
|
||||||
|
|||||||
45
infra/scripts/init-vault.sh
Normal file
45
infra/scripts/init-vault.sh
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Load environment variables
|
||||||
|
source infra/environments/production/.env
|
||||||
|
|
||||||
|
VAULT_ADDR="http://127.0.0.1:8200"
|
||||||
|
CONTAINER_NAME="apa-vault"
|
||||||
|
KEYS_FILE="infra/environments/production/.vault-keys"
|
||||||
|
|
||||||
|
echo "Checking Vault status..."
|
||||||
|
|
||||||
|
# Helper function to run vault commands inside docker
|
||||||
|
vault_cmd() {
|
||||||
|
docker exec -i -e VAULT_ADDR=$VAULT_ADDR $CONTAINER_NAME vault "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if Vault is initialized
|
||||||
|
if vault_cmd status -format=json | grep -q '"initialized": true'; then
|
||||||
|
echo "Vault is already initialized."
|
||||||
|
else
|
||||||
|
echo "Vault is NOT initialized. Initializing..."
|
||||||
|
INIT_OUTPUT=$(vault_cmd operator init -key-shares=1 -key-threshold=1 -format=json)
|
||||||
|
|
||||||
|
echo "$INIT_OUTPUT" > "$KEYS_FILE"
|
||||||
|
chmod 600 "$KEYS_FILE"
|
||||||
|
|
||||||
|
echo "Vault initialized! Keys saved to $KEYS_FILE"
|
||||||
|
echo "WARNING: BACK UP THIS FILE SECURELY!"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Read keys
|
||||||
|
# Extract first key from the array (assuming 1 key share)
|
||||||
|
UNSEAL_KEY=$(grep -A 1 '"unseal_keys_b64":' "$KEYS_FILE" | tail -n 1 | cut -d'"' -f2)
|
||||||
|
ROOT_TOKEN=$(grep '"root_token":' "$KEYS_FILE" | cut -d'"' -f4)
|
||||||
|
|
||||||
|
# Unseal
|
||||||
|
echo "Unsealing Vault..."
|
||||||
|
vault_cmd operator unseal "$UNSEAL_KEY"
|
||||||
|
|
||||||
|
echo "Vault is Unsealed!"
|
||||||
|
echo "Root Token: $ROOT_TOKEN"
|
||||||
|
|
||||||
|
# Export Root Token for setup script
|
||||||
|
export VAULT_TOKEN=$ROOT_TOKEN
|
||||||
84
infra/scripts/setup-vault.sh
Normal file
84
infra/scripts/setup-vault.sh
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Load environment variables
|
||||||
|
source infra/environments/production/.env
|
||||||
|
|
||||||
|
# Vault Configuration
|
||||||
|
VAULT_ADDR="http://127.0.0.1:8200"
|
||||||
|
KEYS_FILE="infra/environments/production/.vault-keys"
|
||||||
|
|
||||||
|
if [ ! -f "$KEYS_FILE" ]; then
|
||||||
|
echo "Error: Keys file not found at $KEYS_FILE. Run init-vault.sh first."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
VAULT_TOKEN=$(grep '"root_token":' "$KEYS_FILE" | cut -d'"' -f4)
|
||||||
|
CONTAINER_NAME="apa-vault"
|
||||||
|
|
||||||
|
echo "Configuring Vault..."
|
||||||
|
|
||||||
|
# Helper function to run vault commands inside docker
|
||||||
|
vault_cmd() {
|
||||||
|
docker exec -i -e VAULT_ADDR=$VAULT_ADDR -e VAULT_TOKEN=$VAULT_TOKEN $CONTAINER_NAME vault "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Enable OIDC auth method
|
||||||
|
echo "Enabling OIDC auth method..."
|
||||||
|
if ! vault_cmd auth list | grep -q "oidc/"; then
|
||||||
|
vault_cmd auth enable oidc
|
||||||
|
else
|
||||||
|
echo "OIDC auth method already enabled."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Configure OIDC
|
||||||
|
echo "Configuring OIDC..."
|
||||||
|
vault_cmd write auth/oidc/config \
|
||||||
|
oidc_discovery_url="https://auth.${DOMAIN}/application/o/vault-prod/" \
|
||||||
|
oidc_client_id="vault-prod" \
|
||||||
|
oidc_client_secret="${AUTHENTIK_VAULT_CLIENT_SECRET}" \
|
||||||
|
default_role="reader"
|
||||||
|
|
||||||
|
# Create Policies
|
||||||
|
echo "Creating policies..."
|
||||||
|
|
||||||
|
# Admin Policy
|
||||||
|
vault_cmd policy write admin - <<EOF
|
||||||
|
path "*" {
|
||||||
|
capabilities = ["create", "read", "update", "delete", "list", "sudo"]
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Reader Policy
|
||||||
|
vault_cmd policy write reader - <<EOF
|
||||||
|
path "secret/*" {
|
||||||
|
capabilities = ["read", "list"]
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Create Roles
|
||||||
|
echo "Creating roles..."
|
||||||
|
|
||||||
|
# Admin Role
|
||||||
|
vault_cmd write auth/oidc/role/admin \
|
||||||
|
bound_audiences="vault-prod" \
|
||||||
|
allowed_redirect_uris="https://vault.${DOMAIN}/ui/vault/auth/oidc/oidc/callback" \
|
||||||
|
allowed_redirect_uris="https://vault.${DOMAIN}/oidc/callback" \
|
||||||
|
user_claim="email" \
|
||||||
|
policies="admin" \
|
||||||
|
role_type="oidc" \
|
||||||
|
groups_claim="groups" \
|
||||||
|
oidc_scopes="openid,email,profile,groups"
|
||||||
|
|
||||||
|
# Reader Role
|
||||||
|
vault_cmd write auth/oidc/role/reader \
|
||||||
|
bound_audiences="vault-prod" \
|
||||||
|
allowed_redirect_uris="https://vault.${DOMAIN}/ui/vault/auth/oidc/oidc/callback" \
|
||||||
|
allowed_redirect_uris="https://vault.${DOMAIN}/oidc/callback" \
|
||||||
|
user_claim="email" \
|
||||||
|
policies="reader" \
|
||||||
|
role_type="oidc" \
|
||||||
|
groups_claim="groups" \
|
||||||
|
oidc_scopes="openid,email,profile,groups"
|
||||||
|
|
||||||
|
echo "Vault configuration complete!"
|
||||||
Reference in New Issue
Block a user