Some checks failed
CI/CD Pipeline / Generate SBOM (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
CI/CD Pipeline / Policy Validation (push) Has been cancelled
CI/CD Pipeline / Test Suite (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-firm-connectors) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-forms) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-hmrc) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ingestion) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-normalize-map) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-ocr) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-indexer) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-reason) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (svc-rpa) (push) Has been cancelled
CI/CD Pipeline / Build Docker Images (ui-review) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-coverage) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-extract) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-kg) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (svc-rag-retriever) (push) Has been cancelled
CI/CD Pipeline / Security Scanning (ui-review) (push) Has been cancelled
CI/CD Pipeline / Notifications (push) Has been cancelled
417 lines
8.8 KiB
Markdown
417 lines
8.8 KiB
Markdown
# Quick Reference Guide
|
|
|
|
## 🚀 Starting Services
|
|
|
|
### Local Development (No Auth Required)
|
|
|
|
```bash
|
|
# Start infrastructure
|
|
make deploy-infra
|
|
|
|
# Run service locally without authentication
|
|
DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
|
|
|
|
# Test it
|
|
curl http://localhost:8000/healthz
|
|
```
|
|
|
|
### Docker Compose (Full Stack)
|
|
|
|
```bash
|
|
# Start all services
|
|
cd infra/compose
|
|
docker compose up -d
|
|
|
|
# Check status
|
|
docker compose ps
|
|
|
|
# View logs
|
|
docker compose logs -f svc-ingestion
|
|
|
|
# Stop all services
|
|
docker compose down
|
|
```
|
|
|
|
## 🔍 Checking Status
|
|
|
|
### Service Health
|
|
|
|
```bash
|
|
# Check all services
|
|
cd infra/compose
|
|
docker compose ps
|
|
|
|
# Count healthy services
|
|
docker compose ps | grep -c "healthy"
|
|
|
|
# Check specific service
|
|
docker compose ps svc-ingestion
|
|
```
|
|
|
|
### Logs
|
|
|
|
```bash
|
|
# View service logs
|
|
cd infra/compose
|
|
docker compose logs -f SERVICE_NAME
|
|
|
|
# View last 50 lines
|
|
docker compose logs --tail=50 SERVICE_NAME
|
|
|
|
# View logs since 5 minutes ago
|
|
docker compose logs --since 5m SERVICE_NAME
|
|
|
|
# Search logs for errors
|
|
docker compose logs SERVICE_NAME | grep -i error
|
|
```
|
|
|
|
### Health Checks
|
|
|
|
```bash
|
|
# Check Traefik health check status
|
|
cd infra/compose
|
|
docker compose logs traefik --since 5m | grep -i "health"
|
|
|
|
# Should show no errors (only certificate warnings are OK)
|
|
```
|
|
|
|
## 🧪 Testing
|
|
|
|
### Health Endpoints (No Auth Required)
|
|
|
|
```bash
|
|
# Health check
|
|
curl http://localhost:8000/healthz
|
|
|
|
# Readiness check
|
|
curl http://localhost:8000/readyz
|
|
|
|
# Liveness check
|
|
curl http://localhost:8000/livez
|
|
|
|
# API documentation
|
|
curl http://localhost:8000/docs
|
|
```
|
|
|
|
### Protected Endpoints (Auth Required)
|
|
|
|
```bash
|
|
# With authentication headers
|
|
curl -X POST http://localhost:8000/upload \
|
|
-H "X-Authenticated-User: dev-user" \
|
|
-H "X-Authenticated-Email: dev@example.com" \
|
|
-H "Authorization: Bearer dev-token-12345" \
|
|
-F "file=@document.pdf"
|
|
```
|
|
|
|
### Development Mode (No Auth Required)
|
|
|
|
```bash
|
|
# When running with DISABLE_AUTH=true
|
|
curl -X POST http://localhost:8000/upload \
|
|
-F "file=@document.pdf"
|
|
```
|
|
|
|
## 🔧 Troubleshooting
|
|
|
|
### Service Won't Start
|
|
|
|
```bash
|
|
# Check logs for errors
|
|
cd infra/compose
|
|
docker compose logs SERVICE_NAME --tail=100
|
|
|
|
# Restart service
|
|
docker compose restart SERVICE_NAME
|
|
|
|
# Rebuild and restart
|
|
docker compose up -d --build SERVICE_NAME
|
|
```
|
|
|
|
### Infrastructure Issues
|
|
|
|
```bash
|
|
# Check infrastructure services
|
|
cd infra/compose
|
|
docker compose ps postgres redis minio neo4j
|
|
|
|
# Restart infrastructure
|
|
docker compose restart postgres redis minio neo4j
|
|
|
|
# Check connectivity
|
|
docker compose exec svc-ingestion ping -c 3 postgres
|
|
```
|
|
|
|
### Health Check Failures
|
|
|
|
```bash
|
|
# Check Traefik logs
|
|
cd infra/compose
|
|
docker compose logs traefik --tail=100 | grep -i "health\|error"
|
|
|
|
# Test health endpoint directly
|
|
docker compose exec SERVICE_NAME curl -f http://localhost:8000/healthz
|
|
|
|
# Restart Traefik
|
|
docker compose restart traefik
|
|
```
|
|
|
|
### Authentication Issues
|
|
|
|
```bash
|
|
# For local development, disable auth
|
|
DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
|
|
|
|
# Check if auth is disabled in logs
|
|
# Should see: "Development mode: authentication disabled"
|
|
|
|
# For production mode, ensure headers are set
|
|
curl -v http://localhost:8000/upload \
|
|
-H "X-Authenticated-User: dev-user" \
|
|
-H "X-Authenticated-Email: dev@example.com" \
|
|
-H "Authorization: Bearer dev-token-12345"
|
|
```
|
|
|
|
## 📊 Monitoring
|
|
|
|
### Service Metrics
|
|
|
|
```bash
|
|
# Prometheus
|
|
open http://localhost:9090
|
|
|
|
# Grafana
|
|
open http://localhost:3000
|
|
|
|
# Traefik Dashboard
|
|
open http://localhost:8080
|
|
```
|
|
|
|
### Database Access
|
|
|
|
```bash
|
|
# PostgreSQL
|
|
docker compose exec postgres psql -U postgres
|
|
|
|
# Redis
|
|
docker compose exec redis redis-cli
|
|
|
|
# Neo4j Browser
|
|
open http://localhost:7474
|
|
```
|
|
|
|
## 🛠️ Common Tasks
|
|
|
|
### Restart All Services
|
|
|
|
```bash
|
|
cd infra/compose
|
|
docker compose restart
|
|
```
|
|
|
|
### Restart Single Service
|
|
|
|
```bash
|
|
cd infra/compose
|
|
docker compose restart svc-ingestion
|
|
```
|
|
|
|
### View Service Configuration
|
|
|
|
```bash
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml config | grep -A 20 "svc-ingestion:"
|
|
```
|
|
|
|
### Clean Up
|
|
|
|
```bash
|
|
# Stop all services
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml down
|
|
|
|
# Stop and remove volumes (⚠️ deletes data)
|
|
docker-compose -f docker-compose.local.yml down -v
|
|
|
|
# Remove all containers and images
|
|
docker-compose -f docker-compose.local.yml down --rmi all
|
|
```
|
|
|
|
### Update Service
|
|
|
|
```bash
|
|
# Rebuild and restart
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml up -d --build svc-ingestion
|
|
|
|
# View logs
|
|
docker-compose -f docker-compose.local.yml logs -f svc-ingestion
|
|
```
|
|
|
|
## 🔐 Environment Variables
|
|
|
|
### Development Mode
|
|
|
|
```bash
|
|
# Disable authentication
|
|
export DISABLE_AUTH=true
|
|
|
|
# Enable development mode
|
|
export DEV_MODE=true
|
|
|
|
# Run service
|
|
make dev-service SERVICE=svc_ingestion
|
|
```
|
|
|
|
### Production Mode
|
|
|
|
```bash
|
|
# Enable authentication (default)
|
|
unset DISABLE_AUTH
|
|
unset DEV_MODE
|
|
|
|
# Run service
|
|
make dev-service SERVICE=svc_ingestion
|
|
```
|
|
|
|
## 📝 Postman
|
|
|
|
### Quick Setup
|
|
|
|
1. **Create Environment**: "AI Tax Agent - Development"
|
|
2. **Add Variables**:
|
|
|
|
- `base_url`: `http://localhost:8000`
|
|
- `auth_user`: `dev-user`
|
|
- `auth_email`: `dev@example.com`
|
|
- `auth_token`: `Bearer dev-token-12345`
|
|
|
|
3. **For Development Mode**: No headers needed
|
|
4. **For Production Mode**: Add headers:
|
|
- `X-Authenticated-User`: `{{auth_user}}`
|
|
- `X-Authenticated-Email`: `{{auth_email}}`
|
|
- `Authorization`: `{{auth_token}}`
|
|
|
|
See [POSTMAN_SETUP.md](POSTMAN_SETUP.md) for detailed instructions.
|
|
|
|
## 📚 Documentation
|
|
|
|
- **[DEVELOPMENT.md](DEVELOPMENT.md)** - Complete development guide
|
|
- **[INFRASTRUCTURE_STATUS.md](INFRASTRUCTURE_STATUS.md)** - Infrastructure status report
|
|
- **[POSTMAN_SETUP.md](POSTMAN_SETUP.md)** - Postman setup guide
|
|
- **[FIXES_APPLIED.md](FIXES_APPLIED.md)** - Recent fixes and changes
|
|
|
|
## 🆘 Getting Help
|
|
|
|
### Check Service Status
|
|
|
|
```bash
|
|
# All services
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml ps
|
|
|
|
# Specific service
|
|
docker-compose -f docker-compose.local.yml ps svc-ingestion
|
|
```
|
|
|
|
### Check Logs
|
|
|
|
```bash
|
|
# Recent logs
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml logs --tail=100 svc-ingestion
|
|
|
|
# Follow logs
|
|
docker-compose -f docker-compose.local.yml logs -f svc-ingestion
|
|
```
|
|
|
|
### Check Health
|
|
|
|
```bash
|
|
# Health endpoint
|
|
curl http://localhost:8000/healthz
|
|
|
|
# Docker health check
|
|
cd infra/compose
|
|
docker-compose -f docker-compose.local.yml ps | grep svc-ingestion
|
|
```
|
|
|
|
### Common Issues
|
|
|
|
| Issue | Solution |
|
|
| -------------------- | ------------------------------------------------- |
|
|
| 401 Unauthorized | Use `DISABLE_AUTH=true` or add auth headers |
|
|
| Connection refused | Check service is running: `docker-compose ps` |
|
|
| 500 Internal Error | Check logs: `docker-compose logs SERVICE_NAME` |
|
|
| Health check failing | Check Traefik logs: `docker-compose logs traefik` |
|
|
| Port already in use | Stop conflicting service or change port |
|
|
|
|
## 🎯 Quick Commands
|
|
|
|
```bash
|
|
# Start everything
|
|
make deploy-infra && DISABLE_AUTH=true make dev-service SERVICE=svc_ingestion
|
|
|
|
# Check status
|
|
curl http://localhost:8000/healthz
|
|
|
|
# View logs
|
|
cd infra/compose && docker-compose -f docker-compose.local.yml logs -f svc-ingestion
|
|
|
|
# Restart service
|
|
cd infra/compose && docker-compose -f docker-compose.local.yml restart svc-ingestion
|
|
|
|
# Stop everything
|
|
cd infra/compose && docker-compose -f docker-compose.local.yml down
|
|
```
|
|
|
|
## 🔄 Service Ports
|
|
|
|
| Service | Port | Access |
|
|
| ----------------- | ---- | --------------------- |
|
|
| svc-ingestion | 8000 | http://localhost:8000 |
|
|
| PostgreSQL | 5432 | localhost:5432 |
|
|
| Redis | 6379 | localhost:6379 |
|
|
| MinIO Console | 9093 | http://localhost:9093 |
|
|
| MinIO API | 9092 | http://localhost:9092 |
|
|
| Neo4j Browser | 7474 | http://localhost:7474 |
|
|
| Neo4j Bolt | 7687 | bolt://localhost:7687 |
|
|
| Qdrant | 6333 | http://localhost:6333 |
|
|
| NATS | 4222 | nats://localhost:4222 |
|
|
| Prometheus | 9090 | http://localhost:9090 |
|
|
| Grafana | 3000 | http://localhost:3000 |
|
|
| Traefik Dashboard | 8080 | http://localhost:8080 |
|
|
| Vault | 8200 | http://localhost:8200 |
|
|
| Unleash | 4242 | http://localhost:4242 |
|
|
|
|
## ✅ Health Check
|
|
|
|
Run this to verify everything is working:
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
echo "🔍 Checking infrastructure..."
|
|
cd infra/compose
|
|
|
|
# Check services
|
|
HEALTHY=$(docker-compose -f docker-compose.local.yml ps | grep -c "healthy")
|
|
echo "✅ Healthy services: $HEALTHY"
|
|
|
|
# Check Traefik
|
|
ERRORS=$(docker-compose -f docker-compose.local.yml logs traefik --since 5m | grep -c "Health check failed")
|
|
if [ $ERRORS -eq 0 ]; then
|
|
echo "✅ No health check errors"
|
|
else
|
|
echo "❌ Found $ERRORS health check errors"
|
|
fi
|
|
|
|
# Test endpoint
|
|
if curl -s http://localhost:8000/healthz > /dev/null; then
|
|
echo "✅ Service responding"
|
|
else
|
|
echo "❌ Service not responding"
|
|
fi
|
|
```
|
|
|
|
Save this as `check-health.sh` and run with `bash check-health.sh`
|