Initial commit

docs/DEVELOPMENT.md

@@ -0,0 +1,240 @@
+# Development Guide
+
+## Running Services Locally
+
+This guide explains how to run services locally for development.
+
+### Prerequisites
+
+1. **Infrastructure Services Running**: Ensure Docker Compose infrastructure is running:
+   ```bash
+   make deploy-infra
+   ```
+
+2. **Python Environment**: Python 3.12+ with virtual environment:
+   ```bash
+   python -m venv .venv
+   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+   pip install -r apps/svc_ingestion/requirements.txt -r libs/requirements.txt
+   ```
+
+### Running a Service in Development Mode
+
+#### Option 1: Using Make (Recommended)
+
+```bash
+# Run with authentication disabled for local development
+DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
+```
+
+#### Option 2: Direct Uvicorn
+
+```bash
+# Navigate to project root
+cd /path/to/ai-tax-agent
+
+# Run with authentication disabled
+DISABLE_AUTH=true cd apps/svc_ingestion && uvicorn main:app --reload --host 0.0.0.0 --port 8000
+```
+
+### Environment Variables for Development
+
+| Variable | Description | Default | Dev Value |
+|----------|-------------|---------|-----------|
+| `DISABLE_AUTH` | Disable authentication middleware | `false` | `true` |
+| `DEV_MODE` | Enable development mode | `false` | `true` |
+| `VAULT_ADDR` | Vault server address | `http://vault:8200` | - |
+| `VAULT_TOKEN` | Vault token (dev only) | - | `root` |
+| `MINIO_ENDPOINT` | MinIO endpoint | `minio:9000` | `minio:9092` |
+| `POSTGRES_URL` | PostgreSQL connection URL | - | `postgresql://postgres:postgres@localhost:5432/tax_system` |
+| `REDIS_URL` | Redis connection URL | `redis://redis:6379` | `redis://localhost:6379` |
+| `NEO4J_URI` | Neo4j connection URI | `bolt://neo4j:7687` | `bolt://localhost:7687` |
+| `NATS_SERVERS` | NATS server URLs | `nats://nats:4222` | `nats://localhost:4222` |
+
+### Testing with Postman
+
+When `DISABLE_AUTH=true` is set, the service runs in development mode and doesn't require authentication headers.
+
+#### Without Development Mode (Production-like)
+
+Add these headers to all requests:
+
+```
+X-Authenticated-User: dev-user
+X-Authenticated-Email: dev@example.com
+Authorization: Bearer dev-token-12345
+```
+
+#### With Development Mode (DISABLE_AUTH=true)
+
+No authentication headers required! The middleware automatically sets:
+- User: `dev-user`
+- Email: `dev@example.com`
+- Roles: `["developers"]`
+- Token: `dev-token`
+
+### Postman Environment Setup
+
+Create a Postman environment called "AI Tax Agent - Dev":
+
+```json
+{
+  "name": "AI Tax Agent - Dev",
+  "values": [
+    {
+      "key": "base_url",
+      "value": "http://localhost:8000",
+      "enabled": true
+    },
+    {
+      "key": "auth_user",
+      "value": "dev-user",
+      "enabled": true
+    },
+    {
+      "key": "auth_email",
+      "value": "dev@example.com",
+      "enabled": true
+    },
+    {
+      "key": "auth_token",
+      "value": "Bearer dev-token-12345",
+      "enabled": true
+    }
+  ]
+}
+```
+
+### Available Endpoints
+
+#### Public Endpoints (No Auth Required)
+
+- `GET /healthz` - Health check
+- `GET /readyz` - Readiness check
+- `GET /livez` - Liveness check
+- `GET /docs` - Swagger UI documentation
+- `GET /openapi.json` - OpenAPI specification
+
+#### Protected Endpoints (Auth Required in Production)
+
+- `POST /upload` - Upload document (requires file in form-data)
+- Service-specific endpoints (see `/docs` for full list)
+
+### Example Requests
+
+#### Health Check
+```bash
+curl http://localhost:8000/healthz
+```
+
+#### Upload Document (Development Mode)
+```bash
+curl -X POST http://localhost:8000/upload \
+  -F "file=@/path/to/document.pdf"
+```
+
+#### Upload Document (Production Mode)
+```bash
+curl -X POST http://localhost:8000/upload \
+  -H "X-Authenticated-User: dev-user" \
+  -H "X-Authenticated-Email: dev@example.com" \
+  -H "Authorization: Bearer dev-token-12345" \
+  -F "file=@/path/to/document.pdf"
+```
+
+### Debugging
+
+#### Check Service Logs
+```bash
+# Local development
+# Logs appear in terminal where service is running
+
+# Docker Compose
+docker-compose -f infra/compose/docker-compose.local.yml logs -f svc-ingestion
+```
+
+#### Verify Infrastructure Services
+```bash
+# Check all services status
+docker-compose -f infra/compose/docker-compose.local.yml ps
+
+# Check specific service health
+docker-compose -f infra/compose/docker-compose.local.yml exec postgres pg_isready
+docker-compose -f infra/compose/docker-compose.local.yml exec redis redis-cli ping
+docker-compose -f infra/compose/docker-compose.local.yml exec minio mc --version
+```
+
+#### Common Issues
+
+**Issue**: `401 Unauthorized` errors
+- **Solution**: Set `DISABLE_AUTH=true` when running locally, or add authentication headers
+
+**Issue**: `Connection refused` to database/redis/etc
+- **Solution**: Ensure infrastructure services are running with `make deploy-infra`
+- **Solution**: Use `localhost` instead of service names when running locally
+
+**Issue**: `Module not found` errors
+- **Solution**: Ensure you're running from project root and virtual environment is activated
+- **Solution**: Install dependencies: `pip install -r apps/SERVICE_NAME/requirements.txt -r libs/requirements.txt`
+
+### Hot Reload
+
+When running with `uvicorn --reload`, the service automatically reloads when you save changes to:
+- Python files in `apps/SERVICE_NAME/`
+- Python files in `libs/`
+
+### Running Multiple Services
+
+To run multiple services simultaneously for integration testing:
+
+```bash
+# Terminal 1: Run ingestion service
+DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
+
+# Terminal 2: Run extraction service  
+DISABLE_AUTH=true make dev-service SERVICE=svc_extract
+
+# Terminal 3: Run knowledge graph service
+DISABLE_AUTH=true make dev-service SERVICE=svc_kg
+```
+
+Each service runs on port 8000 by default, so you'll need to modify the port for additional services:
+
+```bash
+# Terminal 2: Run on port 8001
+DISABLE_AUTH=true cd apps/svc_extract && uvicorn main:app --reload --host 0.0.0.0 --port 8001
+```
+
+### Docker Compose Services
+
+All Docker Compose services are configured with health checks and should show as `healthy`:
+
+```bash
+$ docker-compose -f infra/compose/docker-compose.local.yml ps
+NAME                  STATUS
+authentik-db          Up 35 hours (healthy)
+authentik-outpost     Up 35 hours (healthy)
+authentik-redis       Up 35 hours (healthy)
+authentik-server      Up 35 hours (healthy)
+authentik-worker      Up 35 hours (healthy)
+grafana               Up 35 hours
+loki                  Up 35 hours
+minio                 Up 35 hours (healthy)
+nats                  Up 35 hours (healthy)
+neo4j                 Up 35 hours
+postgres              Up 35 hours (healthy)
+prometheus            Up 35 hours
+qdrant                Up 35 hours
+redis                 Up 35 hours (healthy)
+svc-*                 Up 35 hours (healthy)  # All application services
+traefik               Up 35 hours
+unleash               Up 35 hours
+vault                 Up 35 hours
+```
+
+### Next Steps
+
+- See [README.md](README.md) for architecture overview
+- See [TESTING.md](TESTING.md) for testing guidelines (if available)
+- See service-specific README files in `apps/SERVICE_NAME/` directories
+