roa2web-service-auto/docs/DEPLOYMENT_GUIDE.md

# ROA2WEB Production Deployment Guide

This comprehensive guide covers production deployment of ROA2WEB using Docker, including SSL setup, monitoring, and maintenance procedures.

## 🎯 Overview

ROA2WEB supports two production deployment architectures:

### 🐧 Linux Deployment (Docker)

Multi-container Docker architecture:

- **roa-backend**: FastAPI application server
- **roa-frontend**: Vue.js + Nginx static file server
- **roa-gateway**: Main Nginx reverse proxy with SSL termination
- **roa-redis**: Redis cache and session storage

### 🪟 Windows Server Deployment (IIS)

Native Windows deployment without Docker:

- **IIS Web Server**: Serves frontend static files (port 80/443)
- **Windows Service**: FastAPI backend via NSSM (port 8000)
- **Direct Oracle Connection**: No SSH tunnel required
- **URL Rewrite Module**: Reverse proxy for API routes

---

**Choose your deployment platform:**

- **For Linux servers** → Continue with this guide
- **For Windows Server/IIS** → See [Windows Deployment Guide](#windows-server-deployment-guide)

---

## 📋 Pre-Deployment Checklist (Linux/Docker)

### Infrastructure Requirements

- [ ] **Server**: Linux server with 4GB+ RAM, 20GB+ disk
- [ ] **Docker**: Docker 20.10+ and Docker Compose 2.0+
- [ ] **Domain**: Valid domain name pointing to server IP
- [ ] **Ports**: 80 (HTTP) and 443 (HTTPS) open in firewall
- [ ] **Database**: Oracle database accessible (via SSH tunnel if needed)
- [ ] **SSL**: Email address for Let's Encrypt certificates

### Security Requirements

- [ ] **User**: Non-root user with Docker permissions
- [ ] **SSH**: SSH key-based authentication configured
- [ ] **Firewall**: UFW or iptables configured properly
- [ ] **Updates**: System packages up to date
- [ ] **Backup**: Backup strategy planned and tested

## 🚀 Initial Production Setup

### 1. Server Preparation

```bash
# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Add user to docker group
sudo usermod -aG docker $USER
# Log out and log back in for group changes to take effect

# Install additional tools
sudo apt install -y curl wget git htop ufw
```

### 2. Firewall Configuration

```bash
# Enable UFW
sudo ufw enable

# Allow SSH (adjust port if needed)
sudo ufw allow 22/tcp

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Check status
sudo ufw status
```

### 3. Application Deployment

```bash
# Clone repository
git clone <your-repo-url> /opt/roa2web
cd /opt/roa2web

# Set proper ownership
sudo chown -R $USER:$USER /opt/roa2web

# Copy production environment template
cp .env.production .env.production.local
```

### 4. Production Configuration

Edit `.env.production.local` with your production values:

```env
# Application Environment
ENVIRONMENT=production
DEBUG=false
NODE_ENV=production

# Your Domain
DOMAIN=roa2web.your-domain.com
SSL_EMAIL=admin@your-domain.com

# Database Configuration
ORACLE_USER=CONTAFIN_ORACLE
ORACLE_HOST=your-oracle-server.com
ORACLE_PORT=1521
ORACLE_SID=ROA

# Performance Settings
WORKERS=4
MAX_CONNECTIONS=1000
```

### 5. Secrets Configuration

```bash
# Create secrets directory
mkdir -p secrets/

# Create production secrets (replace with actual values)
echo "<your-database-password>" > secrets/db_password.txt
echo "$(openssl rand -base64 32)" > secrets/jwt_secret_key.txt
echo "$(openssl rand -base64 32)" > secrets/redis_password.txt

# Secure secrets
chmod 600 secrets/*.txt
chmod 700 secrets/
```

### 6. SSL Certificate Setup

```bash
# Generate DH parameters (takes a few minutes)
sudo openssl dhparam -out nginx/ssl/dhparam.pem 2048

# Set proper permissions
chmod 644 nginx/ssl/dhparam.pem
```

## 🔧 Production Deployment

### Method 1: Using Deployment Script (Recommended)

```bash
# Make script executable
chmod +x scripts/deploy.sh

# Deploy to production
./scripts/deploy.sh deploy
```

### Method 2: Manual Deployment

```bash
# Load production environment
set -a && source .env.production.local && set +a

# Build and deploy
docker-compose -f docker-compose.yml -f docker-compose.production.yml up -d --build

# Wait for services to start
sleep 30

# Check health
./scripts/health-check.sh
```

## 🔒 SSL Certificate Management

### Automatic SSL with Let's Encrypt

The deployment automatically handles SSL certificates:

```bash
# Check certificate status
docker-compose exec roa-gateway certbot certificates

# Manual certificate renewal (if needed)
docker-compose exec roa-gateway certbot renew

# Test certificate renewal
docker-compose exec roa-gateway certbot renew --dry-run
```

### Custom SSL Certificates

If using custom certificates:

```bash
# Place certificates in nginx/ssl/
cp your-cert.pem nginx/ssl/
cp your-key.pem nginx/ssl/

# Update nginx/conf/ssl.conf with certificate paths
ssl_certificate /etc/nginx/ssl/your-cert.pem;
ssl_certificate_key /etc/nginx/ssl/your-key.pem;

# Restart gateway
docker-compose restart roa-gateway
```

## 📊 Post-Deployment Verification

### 1. Health Checks

```bash
# Comprehensive health check
./scripts/health-check.sh full

# Check specific endpoints
curl -f https://your-domain.com/health
curl -f https://your-domain.com/api/health

# Check SSL certificate
curl -vI https://your-domain.com 2>&1 | grep -i "ssl certificate"
```

### 2. Performance Testing

```bash
# Simple load test
ab -n 1000 -c 10 https://your-domain.com/

# API load test
ab -n 100 -c 5 https://your-domain.com/api/health

# Monitor during test
./scripts/health-check.sh watch
```

### 3. Security Verification

```bash
# SSL Labs test (external)
# Visit: https://www.ssllabs.com/ssltest/analyze.html?d=your-domain.com

# Check security headers
curl -I https://your-domain.com

# Check for vulnerabilities
docker scout cves backend:latest
```

## 🔄 Maintenance Procedures

### Regular Maintenance Schedule

#### Daily
- [ ] Check service health: `./scripts/health-check.sh quick`
- [ ] Monitor disk space: `df -h`
- [ ] Check error logs: `docker-compose logs --tail=100 | grep -i error`

#### Weekly  
- [ ] Full health check: `./scripts/health-check.sh full`
- [ ] Create backup: `./scripts/backup.sh full`
- [ ] Update system packages: `sudo apt update && sudo apt upgrade`
- [ ] Clean Docker images: `docker system prune -f`

#### Monthly
- [ ] Security audit: `docker scout cves`
- [ ] SSL certificate check: `certbot certificates`
- [ ] Performance review: Load testing
- [ ] Backup validation: Test restore process

### Updates and Rollbacks

#### Application Updates

```bash
# Pull latest code
git pull origin main

# Deploy with backup
./scripts/deploy.sh deploy

# If issues occur, rollback
./scripts/rollback.sh quick
```

#### Rolling Back Deployments

```bash
# Quick rollback (uses previous Docker images)
./scripts/rollback.sh quick

# Full rollback (restores from backup)
./scripts/rollback.sh full

# Rollback to specific backup
./scripts/rollback.sh full backup_20240131_143022
```

### Backup Management

```bash
# Create full backup
./scripts/backup.sh full

# List available backups
./scripts/backup.sh list

# Clean old backups
./scripts/backup.sh cleanup

# Test restore process (in staging)
./scripts/backup.sh restore backup_20240131_143022
```

## 🚨 Monitoring and Alerting

### Log Management

```bash
# View real-time logs
docker-compose logs -f

# Export logs for analysis
docker-compose logs --since="24h" > logs_$(date +%Y%m%d).log

# Search for errors
docker-compose logs | grep -i "error\|exception\|failed"
```

### Performance Monitoring

```bash
# Monitor resource usage
docker stats

# Check system resources
htop

# Monitor disk usage
du -sh /var/lib/docker/
docker system df
```

### Setting up Monitoring (Optional)

For production monitoring, consider adding:

```yaml
# Add to docker-compose.production.yml
services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./monitoring/prometheus.yml:/etc/prometheus/prometheus.yml
    networks:
      - roa-network

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3001:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
    volumes:
      - grafana-data:/var/lib/grafana
    networks:
      - roa-network
```

## 🆘 Emergency Procedures

### Service Recovery

#### If Services Stop Responding

```bash
# Emergency stop all services
./scripts/rollback.sh emergency

# Restart from scratch
docker-compose down -v
docker system prune -f
docker-compose -f docker-compose.yml -f docker-compose.production.yml up -d --build
```

#### If Database Connection Lost

```bash
# Check SSH tunnel (if used)
./ssh_tunnel.sh status
./ssh_tunnel.sh restart

# Restart backend only
docker-compose restart roa-backend

# Check backend logs
docker-compose logs -f roa-backend
```

#### If SSL Certificate Expired

```bash
# Renew certificate immediately
docker-compose exec roa-gateway certbot renew --force-renewal

# Restart gateway
docker-compose restart roa-gateway

# Verify certificate
curl -vI https://your-domain.com
```

### Disaster Recovery

#### Complete System Failure

1. **Restore from backup on new server**:
   ```bash
   # On new server
   git clone <repo-url> /opt/roa2web
   cd /opt/roa2web
   
   # Copy backup from storage
   scp -r backup_server:/backups/backup_20240131_143022 ./backups/
   
   # Restore
   ./scripts/backup.sh restore backup_20240131_143022
   ```

2. **Update DNS records** to point to new server

3. **Generate new SSL certificates**:
   ```bash
   docker-compose exec roa-gateway certbot --nginx -d your-domain.com
   ```

## 🔧 Troubleshooting Guide

### Common Production Issues

#### 1. High Memory Usage

```bash
# Check memory usage
free -h
docker stats

# Solutions:
# - Reduce worker processes in .env
# - Scale down replicas in production compose
# - Add swap space
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
```

#### 2. Disk Space Issues

```bash
# Check disk usage
df -h
docker system df

# Clean up
docker system prune -f
docker image prune -a -f
./scripts/backup.sh cleanup
```

#### 3. SSL Certificate Issues

```bash
# Check certificate status
certbot certificates

# Debug SSL issues
docker-compose exec roa-gateway nginx -t
docker-compose logs roa-gateway
```

#### 4. High CPU Usage

```bash
# Check processes
htop
docker stats

# Solutions:
# - Check for resource-intensive queries
# - Optimize database connections
# - Scale horizontally
```

## 📞 Support and Escalation

### Getting Help

1. **Check logs**: `docker-compose logs`
2. **Run diagnostics**: `./scripts/health-check.sh full`
3. **Review monitoring**: Check Grafana/Prometheus (if configured)
4. **Check documentation**: This guide and DOCKER_SETUP.md
5. **Contact team**: development-team@your-company.com

### Escalation Procedures

#### Severity 1 (Service Down)
- Immediate response required
- Use emergency procedures
- Contact on-call engineer

#### Severity 2 (Performance Issues)  
- Response within 2 hours
- Investigation and resolution
- Document root cause

#### Severity 3 (Minor Issues)
- Response within 8 hours
- Standard troubleshooting
- Schedule resolution

---

## 🪟 Windows Server Deployment Guide

This section covers production deployment on Windows Server with IIS, without Docker.

### 📦 Windows Prerequisites

**Server Requirements:**

| Component | Requirement | Notes |
|-----------|-------------|-------|
| **OS** | Windows Server 2016+ | Or Windows 10/11 Pro |
| **RAM** | 4GB minimum | 8GB recommended |
| **Disk** | 10GB free space | For application and logs |
| **CPU** | 2 cores minimum | 4 cores recommended |

**Required Software (installed automatically):**

- IIS (Internet Information Services)
- Python 3.11+
- NSSM (Non-Sucking Service Manager)
- IIS URL Rewrite Module
- IIS Application Request Routing (ARR)

**Pre-installed:**

- Oracle Database (local or network-accessible)
- Oracle Instant Client (for Python oracledb)

### 🚀 Windows Installation Steps

#### Step 1: Install IIS

Open PowerShell as Administrator:

```powershell
# Install IIS with required features
Install-WindowsFeature -Name Web-Server -IncludeManagementTools

# Verify installation
Get-WindowsFeature -Name Web-Server
```

#### Step 2: Prepare Deployment Package

**On development machine (WSL/Linux/Mac):**

```bash
# Navigate to deployment scripts
cd /mnt/e/proiecte/deployment/windows/scripts

# Build frontend and create deployment package
./Build-Frontend.ps1

# Output: ./deploy-package/
```

The build creates a complete deployment package:
```
deploy-package/
├── backend/           # Backend files with shared components
├── frontend/          # Built Vue.js files
├── config/            # Configuration templates
├── scripts/           # PowerShell management scripts
└── README.txt         # Deployment instructions
```

#### Step 3: Transfer to Server

**Option A: Network Share**
```powershell
# On development machine
Copy-Item -Path .\deploy-package -Destination \\SERVER-IP\C$\Temp\roa2web -Recurse
```

**Option B: Manual Transfer**
- Zip the `deploy-package` folder
- Transfer via RDP, FTP, or USB
- Extract on server to `C:\Temp\roa2web`

#### Step 4: Run Installation Script

**On Windows Server (PowerShell as Administrator):**

```powershell
# Navigate to the transferred package
cd C:\Temp\roa2web\scripts

# Run installation
.\Install-ROA2WEB.ps1

# Installation will:
# ✅ Install Python, NSSM, IIS modules
# ✅ Create directory structure at C:\inetpub\wwwroot\roa2web\
# ✅ Install Python dependencies
# ✅ Create Windows Service (ROA2WEB-Backend)
# ✅ Configure IIS website (ROA2WEB)
```

**Installation Parameters:**

```powershell
# Custom installation path
.\Install-ROA2WEB.ps1 -InstallPath "D:\Apps\roa2web"

# Custom service port
.\Install-ROA2WEB.ps1 -ServicePort 8001

# Skip Python installation (if already installed)
.\Install-ROA2WEB.ps1 -SkipPython

# Skip IIS configuration
.\Install-ROA2WEB.ps1 -SkipIIS
```

#### Step 5: Configure Application

**Edit configuration file:**

```powershell
# The .env template is already in place
# Edit with your production values
notepad C:\inetpub\wwwroot\roa2web\backend\.env
```

**Required configuration:**

```env
# Oracle Database
ORACLE_USER=CONTAFIN_ORACLE
ORACLE_PASS=<your-db-password>
ORACLE_HOST=localhost
ORACLE_PORT=1521
ORACLE_SID=ROA

# JWT Secret (MUST be changed!)
JWT_SECRET_KEY=GENERATE_STRONG_RANDOM_STRING_HERE
JWT_ALGORITHM=HS256
JWT_EXPIRE_MINUTES=480

# Server Settings
HOST=127.0.0.1
PORT=8000
WORKERS=4

# Logging
LOG_LEVEL=INFO
```

**Generate JWT Secret:**

```powershell
# PowerShell method (copy the output)
-join ((65..90) + (97..122) + (48..57) | Get-Random -Count 32 | % {[char]$_})
```

#### Step 6: Deploy Application Files

```powershell
# Deploy backend and frontend to installation directory
.\Deploy-ROA2WEB.ps1 -SourcePath "C:\Temp\roa2web"

# This will:
# - Create backup of any existing deployment
# - Copy backend and frontend files
# - Install Python dependencies
# - Configure IIS website
```

#### Step 7: Start Services

```powershell
# Start backend service
.\Start-ROA2WEB.ps1

# Check service status
Get-Service ROA2WEB-Backend

# Check IIS website
Get-Website ROA2WEB

# View logs
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stdout.log -Tail 50
```

#### Step 8: Verify Installation

**Test endpoints:**

```powershell
# Backend health check
Invoke-WebRequest -Uri "http://localhost:8000/health"

# API documentation
Start-Process "http://localhost:8000/docs"

# Frontend application
Start-Process "http://localhost"

# Test API through IIS proxy
Invoke-WebRequest -Uri "http://localhost/api/health"
```

### 🔄 Windows Update Workflow

**For updates and new deployments:**

**1. Build on Development Machine:**

```bash
cd /mnt/e/proiecte/deployment/windows/scripts
./Build-Frontend.ps1 -OutputPath "./deploy-$(date +%Y%m%d)"
```

**2. Transfer to Server:**

```powershell
Copy-Item -Path .\deploy-20250118 -Destination C:\Temp\roa2web-update -Recurse
```

**3. Deploy on Server:**

```powershell
cd C:\inetpub\wwwroot\roa2web\deployment\windows\scripts

# Run deployment script
.\Deploy-ROA2WEB.ps1 -SourcePath "C:\Temp\roa2web-update"

# The script will:
# ✅ Create automatic backup
# ✅ Stop backend service
# ✅ Update backend and frontend files
# ✅ Install new dependencies (if changed)
# ✅ Restart backend service
# ✅ Validate deployment health
```

**Deployment Options:**

```powershell
# Update only backend
.\Deploy-ROA2WEB.ps1 -UpdateFrontend $false

# Update only frontend
.\Deploy-ROA2WEB.ps1 -UpdateBackend $false

# Skip backup (not recommended)
.\Deploy-ROA2WEB.ps1 -BackupEnabled $false
```

### 🔧 Windows Management

**Service Management:**

```powershell
# Management scripts
.\Start-ROA2WEB.ps1
.\Stop-ROA2WEB.ps1
.\Restart-ROA2WEB.ps1

# Manual service commands
Start-Service ROA2WEB-Backend
Stop-Service ROA2WEB-Backend
Restart-Service ROA2WEB-Backend
Get-Service ROA2WEB-Backend

# Windows Services GUI
services.msc
```

**Log Management:**

```powershell
# Real-time monitoring
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stdout.log -Tail 50 -Wait

# Last 100 lines
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stderr.log -Tail 100

# Search for errors
Select-String -Path "C:\inetpub\wwwroot\roa2web\logs\*.log" -Pattern "ERROR|CRITICAL"
```

**IIS Management:**

```powershell
# Website status
Get-Website ROA2WEB

# Start/Stop website
Start-Website ROA2WEB
Stop-Website ROA2WEB

# Application pool
Get-WebAppPoolState ROA2WEB-AppPool
Restart-WebAppPool ROA2WEB-AppPool

# IIS Manager GUI
inetmgr
```

### 🐛 Windows Troubleshooting

**Service Won't Start:**

```powershell
# Check error log
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stderr.log -Tail 50

# Test manually
cd C:\inetpub\wwwroot\roa2web\backend
python -m uvicorn app.main:app --host 127.0.0.1 --port 8000

# Check Python installation
python --version

# Reinstall dependencies
pip install -r requirements.txt
```

**Frontend Not Loading:**

```powershell
# Check IIS website status
Get-Website ROA2WEB

# Restart IIS
Stop-Website ROA2WEB
Start-Website ROA2WEB

# Check files exist
Test-Path C:\inetpub\wwwroot\roa2web\frontend\index.html

# Check IIS logs
Get-Content C:\inetpub\logs\LogFiles\W3SVC*\*.log -Tail 50
```

**API Calls Failing (502/504):**

```powershell
# Check backend service
Get-Service ROA2WEB-Backend
.\Restart-ROA2WEB.ps1

# Test backend directly
Invoke-WebRequest -Uri "http://localhost:8000/health"

# Enable ARR proxy (if not enabled)
Set-WebConfigurationProperty -PSPath "MACHINE/WEBROOT/APPHOST" `
    -Filter "system.webServer/proxy" `
    -Name "enabled" `
    -Value "True"
```

**Database Connection Issues:**

```powershell
# Check Oracle service
Get-Service Oracle*

# Test Oracle connection
sqlplus CONTAFIN_ORACLE/password@localhost:1521/ROA

# Verify .env configuration
Get-Content C:\inetpub\wwwroot\roa2web\backend\.env | Select-String ORACLE
```

### 📊 Windows Architecture Diagram

```
Client Browser
    ↓
IIS Web Server (Port 80/443)
    ├─→ /api/* ──[URL Rewrite]──→ Backend Service (localhost:8000)
    │                                    ↓
    │                               Oracle DB (localhost:1521)
    └─→ /* ──[Static Files]──→ Frontend (Vue.js)
```

### 📁 Windows Directory Structure

```
C:\inetpub\wwwroot\roa2web\
├── backend\                      # FastAPI application
│   ├── app\
│   ├── shared\                  # Shared components
│   ├── requirements.txt
│   ├── .env                     # Production config
│   └── logs\
├── frontend\                     # Vue.js static files
│   ├── index.html
│   ├── assets\
│   └── web.config               # IIS configuration
├── logs\                         # Service logs
│   ├── backend-stdout.log
│   └── backend-stderr.log
├── backups\                      # Automatic backups
│   └── backup-YYYYMMDD-HHMMSS\
└── deployment\                   # Deployment scripts
    └── windows\
        ├── scripts\             # PowerShell scripts
        ├── config\              # Configuration templates
        └── docs\                # Documentation
```

### 🔐 Windows Security

**Generate Strong JWT Secret:**

```powershell
-join ((65..90) + (97..122) + (48..57) | Get-Random -Count 32 | % {[char]$_})
```

**Secure .env File:**

```powershell
icacls C:\inetpub\wwwroot\roa2web\backend\.env /inheritance:r /grant:r Administrators:F
```

**Enable HTTPS:**

1. Install SSL certificate in IIS
2. Add HTTPS binding to ROA2WEB website
3. Update web.config with HTTPS redirect rules

### 📚 Windows Deployment Documentation

For complete Windows deployment documentation, see:

- **[/deployment/windows/README.md](./deployment/windows/README.md)** - Quick start guide
- **[/deployment/windows/docs/WINDOWS_DEPLOYMENT.md](./deployment/windows/docs/WINDOWS_DEPLOYMENT.md)** - Complete deployment guide
- **[/deployment/windows/config/](./deployment/windows/config/)** - Configuration templates

---

## 📚 Additional Resources

### Linux/Docker Deployment

- [DOCKER_SETUP.md](./DOCKER_SETUP.md) - Development setup
- [PRODUCTION_CHECKLIST.md](./PRODUCTION_CHECKLIST.md) - Go-live checklist
- [Scripts Documentation](./scripts/) - Deployment scripts
- [Nginx Configuration](./nginx/conf/) - Web server config

### Windows Server Deployment

- [Windows Deployment Guide](./deployment/windows/docs/WINDOWS_DEPLOYMENT.md) - Complete guide
- [Windows README](./deployment/windows/README.md) - Quick reference
- [PowerShell Scripts](./deployment/windows/scripts/) - Automation scripts
- [Configuration Templates](./deployment/windows/config/) - IIS and environment configs

### General Documentation

- [ARCHITECTURE_SCHEMA.md](./ARCHITECTURE_SCHEMA.md) - Architecture overview
- [DEVELOPMENT_BLUEPRINT.md](./DEVELOPMENT_BLUEPRINT.md) - Development guide
- [MICROSERVICES_GUIDE.md](./MICROSERVICES_GUIDE.md) - Microservices architecture

---

*Last updated: 2025-01-18*
*ROA2WEB Production Deployment Guide v2.0 - Linux & Windows*