completed local setup with compose
Some checks failed
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled

This commit is contained in:
harkon
2025-11-26 13:17:17 +00:00
parent 8fe5e62fee
commit fdba81809f
87 changed files with 5610 additions and 3376 deletions

View File

@@ -7,6 +7,7 @@ This guide explains how to run services locally for development.
### Prerequisites
1. **Infrastructure Services Running**: Ensure Docker Compose infrastructure is running:
```bash
make deploy-infra
```
@@ -39,17 +40,17 @@ DISABLE_AUTH=true cd apps/svc_ingestion && uvicorn main:app --reload --host 0.0.
### Environment Variables for Development
| Variable | Description | Default | Dev Value |
|----------|-------------|---------|-----------|
| `DISABLE_AUTH` | Disable authentication middleware | `false` | `true` |
| `DEV_MODE` | Enable development mode | `false` | `true` |
| `VAULT_ADDR` | Vault server address | `http://vault:8200` | - |
| `VAULT_TOKEN` | Vault token (dev only) | - | `root` |
| `MINIO_ENDPOINT` | MinIO endpoint | `minio:9000` | `minio:9092` |
| `POSTGRES_URL` | PostgreSQL connection URL | - | `postgresql://postgres:postgres@localhost:5432/tax_system` |
| `REDIS_URL` | Redis connection URL | `redis://redis:6379` | `redis://localhost:6379` |
| `NEO4J_URI` | Neo4j connection URI | `bolt://neo4j:7687` | `bolt://localhost:7687` |
| `NATS_SERVERS` | NATS server URLs | `nats://nats:4222` | `nats://localhost:4222` |
| Variable | Description | Default | Dev Value |
| ---------------- | --------------------------------- | -------------------- | ---------------------------------------------------------- |
| `DISABLE_AUTH` | Disable authentication middleware | `false` | `true` |
| `DEV_MODE` | Enable development mode | `false` | `true` |
| `VAULT_ADDR` | Vault server address | `http://vault:8200` | - |
| `VAULT_TOKEN` | Vault token (dev only) | - | `root` |
| `MINIO_ENDPOINT` | MinIO endpoint | `minio:9000` | `minio:9092` |
| `POSTGRES_URL` | PostgreSQL connection URL | - | `postgresql://postgres:postgres@localhost:5432/tax_system` |
| `REDIS_URL` | Redis connection URL | `redis://redis:6379` | `redis://localhost:6379` |
| `NEO4J_URI` | Neo4j connection URI | `bolt://neo4j:7687` | `bolt://localhost:7687` |
| `NATS_SERVERS` | NATS server URLs | `nats://nats:4222` | `nats://localhost:4222` |
### Testing with Postman
@@ -68,6 +69,7 @@ Authorization: Bearer dev-token-12345
#### With Development Mode (DISABLE_AUTH=true)
No authentication headers required! The middleware automatically sets:
- User: `dev-user`
- Email: `dev@example.com`
- Roles: `["developers"]`
@@ -123,17 +125,20 @@ Create a Postman environment called "AI Tax Agent - Dev":
### Example Requests
#### Health Check
```bash
curl http://localhost:8000/healthz
```
#### Upload Document (Development Mode)
```bash
curl -X POST http://localhost:8000/upload \
-F "file=@/path/to/document.pdf"
```
#### Upload Document (Production Mode)
```bash
curl -X POST http://localhost:8000/upload \
-H "X-Authenticated-User: dev-user" \
@@ -145,41 +150,47 @@ curl -X POST http://localhost:8000/upload \
### Debugging
#### Check Service Logs
```bash
# Local development
# Logs appear in terminal where service is running
# Docker Compose
docker-compose -f infra/compose/docker-compose.local.yml logs -f svc-ingestion
docker compose logs -f svc-ingestion
```
#### Verify Infrastructure Services
```bash
# Check all services status
docker-compose -f infra/compose/docker-compose.local.yml ps
docker compose ps
# Check specific service health
docker-compose -f infra/compose/docker-compose.local.yml exec postgres pg_isready
docker-compose -f infra/compose/docker-compose.local.yml exec redis redis-cli ping
docker-compose -f infra/compose/docker-compose.local.yml exec minio mc --version
docker compose exec postgres pg_isready
docker compose exec redis redis-cli ping
docker compose exec minio mc --version
```
#### Common Issues
**Issue**: `401 Unauthorized` errors
- **Solution**: Set `DISABLE_AUTH=true` when running locally, or add authentication headers
**Issue**: `Connection refused` to database/redis/etc
- **Solution**: Ensure infrastructure services are running with `make deploy-infra`
- **Solution**: Use `localhost` instead of service names when running locally
**Issue**: `Module not found` errors
- **Solution**: Ensure you're running from project root and virtual environment is activated
- **Solution**: Install dependencies: `pip install -r apps/SERVICE_NAME/requirements.txt -r libs/requirements.txt`
### Hot Reload
When running with `uvicorn --reload`, the service automatically reloads when you save changes to:
- Python files in `apps/SERVICE_NAME/`
- Python files in `libs/`
@@ -191,7 +202,7 @@ To run multiple services simultaneously for integration testing:
# Terminal 1: Run ingestion service
DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
# Terminal 2: Run extraction service
# Terminal 2: Run extraction service
DISABLE_AUTH=true make dev-service SERVICE=svc_extract
# Terminal 3: Run knowledge graph service
@@ -210,7 +221,7 @@ DISABLE_AUTH=true cd apps/svc_extract && uvicorn main:app --reload --host 0.0.0.
All Docker Compose services are configured with health checks and should show as `healthy`:
```bash
$ docker-compose -f infra/compose/docker-compose.local.yml ps
$ docker compose ps
NAME STATUS
authentik-db Up 35 hours (healthy)
authentik-outpost Up 35 hours (healthy)
@@ -237,4 +248,3 @@ vault Up 35 hours
- See [README.md](README.md) for architecture overview
- See [TESTING.md](TESTING.md) for testing guidelines (if available)
- See service-specific README files in `apps/SERVICE_NAME/` directories