# Feature: Ultrathin Monolith Backend Architecture

## Overview

Transform ROA2WEB from 3 separate backend services (reports-app:8001, data-entry-app:8003, telegram-bot:8002) into a single unified backend process running on port 8000. This architectural transformation simplifies deployment, centralizes logs, and reduces operational complexity for solo developer workflow while maintaining clear module boundaries.

## Problem Statement

**Current Pain Points:**
- Running 3 separate backend processes is complex to manage (start/stop/monitor)
- Windows deployment requires 3 NSSM services, significantly increasing deployment time
- Debugging is difficult with fragmented logs across multiple processes
- Resource overhead of multiple Python processes and connection pools
- Frontend already unified into single SPA, but backend remains fragmented

**Who Benefits:**
- Solo developer: Easier development workflow, simplified debugging
- Windows deployment: Single service instead of 3 (faster deployment)
- System resources: Single Oracle pool, single process overhead

## User Stories

- As a developer, I want to start the entire backend with a single command (`python main.py`) so that I can begin working faster
- As a developer, I want centralized logs in a single stream so that I can debug issues across all modules more easily
- As a deployer, I want to install a single Windows service so that deployment takes less time
- As a system administrator, I want a single backend process so that resource usage is more efficient
- As a developer, I want to keep separate module directories (reports/, data-entry/, telegram-bot/) so that code organization remains clear

## Functional Requirements

### Core Requirements

1. **Single Entry Point**
   - One `main.py` file at project root (`/backend/main.py`)
   - Single FastAPI application instance
   - Single uvicorn process running on port 8000

2. **Module Prefix Routing**
   - Reports endpoints: `/api/reports/*` (e.g., `/api/reports/invoices`, `/api/reports/dashboard`)
   - Data Entry endpoints: `/api/data-entry/*` (e.g., `/api/data-entry/receipts`, `/api/data-entry/ocr`)
   - Telegram endpoints: `/api/telegram/*` (e.g., `/api/telegram/auth/verify-user`)
   - Shared endpoints: `/api/auth/*`, `/api/companies`, `/api/calendar` (no prefix change)

3. **Integrated Telegram Bot**
   - Telegram bot runs in same process (background thread/task)
   - Internal API endpoints accessible via `/api/telegram/*`
   - Shared database access (no separate SQLite for bot)

4. **Unified Startup/Shutdown**
   - Single lifespan manager initializing all modules
   - Parallel initialization where possible (Oracle pool + PaddleOCR)
   - Graceful shutdown for all components
   - Centralized logging configuration

5. **Module Directory Structure**
   - Keep existing directories: `backend/modules/reports/`, `backend/modules/data-entry/`, `backend/modules/telegram/`
   - Each module retains its own routers, services, models
   - Shared code remains in `shared/` (auth, database, routes)

### Secondary Requirements

1. **Environment Variable Consolidation**
   - Single `.env` file at `backend/.env`
   - Merge variables from all 3 backends
   - Namespace conflicts resolved (e.g., `REPORTS_*`, `DATA_ENTRY_*` prefixes)

2. **Single Virtual Environment**
   - Merged `requirements.txt` with all dependencies
   - De-duplicated dependencies (single version of FastAPI, etc.)
   - Optional dependency groups for OCR/Telegram

3. **Health Check Aggregation**
   - Single `/health` endpoint reporting status of all modules
   - Check Oracle connection, SQLite database, Telegram bot status
   - Return comprehensive health information

4. **Code Cleanup**
   - Remove duplicate auth router implementations (use shared router factory)
   - Remove duplicate company/calendar router implementations
   - Consolidate logging configuration
   - Remove unused imports and code

## Technical Requirements

### Files to Create

| File | Purpose |
|------|---------|
| `/backend/main.py` | Single unified entry point - creates FastAPI app, registers all module routers, manages lifecycle |
| `/backend/config.py` | Unified configuration management (merges settings from all apps) |
| `/backend/requirements.txt` | Consolidated dependencies from all 3 backends |
| `/backend/.env.example` | Merged environment variables template |
| `/backend/modules/__init__.py` | Module package marker |
| `/backend/modules/reports/__init__.py` | Reports module package |
| `/backend/modules/data-entry/__init__.py` | Data entry module package |
| `/backend/modules/telegram/__init__.py` | Telegram module package |

### Files to Modify

| File | Changes |
|------|---------|
| `reports-app/backend/app/routers/*.py` | Move to `backend/modules/reports/routers/` |
| `reports-app/backend/app/services/*.py` | Move to `backend/modules/reports/services/` |
| `reports-app/backend/app/models/*.py` | Move to `backend/modules/reports/models/` |
| `reports-app/backend/app/cache/**/*.py` | Move to `backend/modules/reports/cache/` (reports-specific caching) |
| `data-entry-app/backend/app/routers/*.py` | Move to `backend/modules/data-entry/routers/` |
| `data-entry-app/backend/app/services/*.py` | Move to `backend/modules/data-entry/services/` |
| `data-entry-app/backend/app/db/**/*.py` | Move to `backend/modules/data-entry/db/` |
| `data-entry-app/backend/migrations/` | Move to `backend/modules/data-entry/migrations/` |
| `reports-app/telegram-bot/app/bot/*.py` | Move to `backend/modules/telegram/bot/` |
| `reports-app/telegram-bot/app/db/*.py` | Move to `backend/modules/telegram/db/` |
| `reports-app/telegram-bot/app/internal_api.py` | Integrate into main FastAPI app as router |
| All router files | Update imports to reflect new paths |
| `vite.config.js` | Update proxy configuration to single backend (port 8000) |
| `public/web.config` (IIS) | Update proxy rules to single backend (port 8000) |

### New Directory Structure

```
roa2web/
├── backend/                         # UNIFIED BACKEND
│   ├── main.py                      # Single entry point
│   ├── config.py                    # Unified configuration
│   ├── requirements.txt             # Merged dependencies
│   ├── .env                         # Merged environment variables
│   ├── .env.example                 # Environment template
│   ├── venv/                        # Single virtual environment
│   │
│   ├── modules/                     # Module-specific code
│   │   ├── __init__.py
│   │   │
│   │   ├── reports/                 # Reports module (from reports-app)
│   │   │   ├── __init__.py
│   │   │   ├── routers/             # API endpoints (invoices, dashboard, treasury, etc.)
│   │   │   ├── services/            # Business logic
│   │   │   ├── models/              # Pydantic models
│   │   │   ├── schemas/             # Response schemas
│   │   │   └── cache/               # Caching system (L1+L2)
│   │   │
│   │   ├── data-entry/              # Data entry module (from data-entry-app)
│   │   │   ├── __init__.py
│   │   │   ├── routers/             # Receipts, OCR, nomenclature routers
│   │   │   ├── services/            # Business logic + OCR service
│   │   │   ├── db/                  # SQLModel models + CRUD
│   │   │   ├── schemas/             # Pydantic schemas
│   │   │   └── migrations/          # Alembic migrations
│   │   │
│   │   └── telegram/                # Telegram bot module (from telegram-bot)
│   │       ├── __init__.py
│   │       ├── bot/                 # Command handlers, formatters
│   │       ├── db/                  # Bot-specific database (auth codes, sessions)
│   │       └── routers/             # Internal API endpoints (auth code management)
│   │
│   └── data/                        # Data files
│       ├── cache/                   # SQLite cache files (reports module)
│       ├── receipts/                # SQLite DB + uploads (data-entry module)
│       └── telegram/                # SQLite DB for bot (telegram module)
│
├── shared/                          # Shared components (unchanged)
│   ├── auth/                        # JWT authentication
│   ├── database/                    # Oracle pool
│   ├── routes/                      # Shared router factories
│   └── frontend/                    # Shared Vue components
│
├── frontend/                        # Unified frontend (unchanged structure)
│   ├── src/
│   │   ├── modules/
│   │   │   ├── reports/
│   │   │   └── data-entry/
│   │   └── shared/
│   └── vite.config.js               # UPDATE: Single proxy to localhost:8000
│
├── deployment/                      # Deployment scripts
│   └── windows/                     # Windows deployment
│       └── install-service.ps1      # UPDATE: Single NSSM service
│
└── docs/                            # Documentation
```

### Dependencies Consolidation

**Common Dependencies** (deduplicated):
```
fastapi>=0.109.0
uvicorn[standard]>=0.27.0
pydantic>=2.5.3
python-dotenv>=1.0.0
PyJWT>=2.8.0
python-jose[cryptography]>=3.3.0
oracledb>=2.0.1
aiosqlite>=0.19.0
httpx>=0.27.0
python-multipart>=0.0.6
```

**Reports-Specific**:
```
openpyxl>=3.1.0              # Excel export
fpdf2>=2.7.0                 # PDF generation
python-dateutil>=2.8.2       # Date utilities
```

**Data Entry-Specific**:
```
sqlmodel>=0.0.14             # ORM
alembic>=1.13.1              # Migrations
aiofiles>=23.2.1             # Async file handling
Pillow>=10.2.0               # Image processing
paddleocr>=2.7.0             # OCR engine
paddlepaddle>=2.5.0          # PaddleOCR backend
opencv-python>=4.8.0         # Image processing
pytesseract>=0.3.10          # Tesseract OCR
pdf2image>=1.16.0            # PDF to image conversion
numpy>=1.24.0                # Array operations
```

**Telegram-Specific**:
```
python-telegram-bot>=20.7    # Telegram bot SDK
aiosmtplib>=3.0.0            # Email (2FA)
sentry-sdk>=1.40.0           # Monitoring (optional)
```

### Database Changes

**No schema changes** - databases remain separate:
- Oracle: Shared via `oracle_pool` (reports + data-entry nomenclatures + auth)
- SQLite (`receipts.db`): Data entry module receipts data
- SQLite (`cache.db`): Reports module caching
- SQLite (`telegram.db`): Telegram bot auth codes and sessions

**Database Access Patterns**:
- Oracle Pool: Single shared instance initialized at startup, used by all modules
- SQLite databases: Separate per module, stored in `backend/data/`

### API Changes

**URL Migration** (frontend must update):

| Old URL | New URL | Module |
|---------|---------|--------|
| `http://localhost:8001/api/invoices` | `http://localhost:8000/api/reports/invoices` | Reports |
| `http://localhost:8001/api/dashboard` | `http://localhost:8000/api/reports/dashboard` | Reports |
| `http://localhost:8001/api/treasury` | `http://localhost:8000/api/reports/treasury` | Reports |
| `http://localhost:8001/api/trial-balance` | `http://localhost:8000/api/reports/trial-balance` | Reports |
| `http://localhost:8001/api/cache/*` | `http://localhost:8000/api/reports/cache/*` | Reports |
| `http://localhost:8003/api/receipts` | `http://localhost:8000/api/data-entry/receipts` | Data Entry |
| `http://localhost:8003/api/ocr` | `http://localhost:8000/api/data-entry/ocr` | Data Entry |
| `http://localhost:8003/api/nomenclature` | `http://localhost:8000/api/data-entry/nomenclature` | Data Entry |
| `http://localhost:8002/api/telegram/*` | `http://localhost:8000/api/telegram/*` | Telegram |
| `http://localhost:8001/api/auth/login` | `http://localhost:8000/api/auth/login` | Shared (no change) |
| `http://localhost:8001/api/companies` | `http://localhost:8000/api/companies` | Shared (no change) |
| `http://localhost:8001/api/calendar` | `http://localhost:8000/api/calendar` | Shared (no change) |

**Frontend Configuration Updates**:

```javascript
// vite.config.js - BEFORE (3 proxies)
proxy: {
  '/api/reports': { target: 'http://localhost:8001', rewrite: (path) => path.replace(/^\/api\/reports/, '/api') },
  '/api/data-entry': { target: 'http://localhost:8003', rewrite: (path) => path.replace(/^\/api\/data-entry/, '/api') },
  '/api/telegram': { target: 'http://localhost:8002', rewrite: (path) => path.replace(/^\/api\/telegram/, '/api') }
}

// vite.config.js - AFTER (1 proxy)
proxy: {
  '/api': { target: 'http://localhost:8000', changeOrigin: true }
}
```

```xml
<!-- public/web.config - BEFORE (3 rules) -->
<rule name="Proxy Reports API" stopProcessing="true">
  <match url="^api/reports/(.*)" />
  <action type="Rewrite" url="http://localhost:8001/api/{R:1}" />
</rule>
<rule name="Proxy Data Entry API" stopProcessing="true">
  <match url="^api/data-entry/(.*)" />
  <action type="Rewrite" url="http://localhost:8003/api/{R:1}" />
</rule>
<rule name="Proxy Telegram API" stopProcessing="true">
  <match url="^api/telegram/(.*)" />
  <action type="Rewrite" url="http://localhost:8002/api/{R:1}" />
</rule>

<!-- public/web.config - AFTER (1 rule) -->
<rule name="Proxy Backend API" stopProcessing="true">
  <match url="^api/(.*)" />
  <action type="Rewrite" url="http://localhost:8000/api/{R:1}" />
</rule>
```

## Design Decisions

### Approach: Module-Based Monolith with Clear Boundaries

**Why This Approach:**
1. **Maintains Module Isolation**: Each module (reports, data-entry, telegram) keeps its own directory structure
2. **Simplifies Deployment**: Single process, single service, single log stream
3. **Preserves Code Organization**: Module boundaries remain clear in file structure
4. **Enables Gradual Migration**: Can move one module at a time
5. **Reduces Complexity**: Solo developer doesn't need microservices orchestration

**Key Architectural Patterns:**
- **Single FastAPI App**: One application instance with multiple routers registered with prefixes
- **Module Routers**: Each module exports a function that returns its configured router
- **Parallel Initialization**: Oracle pool and PaddleOCR load concurrently at startup
- **Unified Logging**: Single logging configuration with module-specific loggers
- **Shared Singleton**: Oracle pool shared across all modules

### Alternatives Considered

**Alternative 1: Keep Microservices Architecture**
- **Rejected**: Adds complexity without benefits for solo developer
- **Rejected**: Windows deployment remains slow (3 services)
- **Rejected**: Debugging remains difficult (fragmented logs)

**Alternative 2: Merge Everything into Flat Structure**
- **Rejected**: Loses module boundaries, code becomes harder to navigate
- **Rejected**: Difficult to understand which code belongs to which feature

**Alternative 3: Use Docker Compose for Orchestration**
- **Rejected**: Adds Docker overhead without solving core issues
- **Rejected**: Windows deployment still needs Docker Desktop or manual setup
- **Rejected**: Doesn't simplify solo developer workflow

## Acceptance Criteria

- [ ] Single `python backend/main.py` command starts entire backend
- [ ] All endpoints accessible on port 8000 with module prefixes (`/api/reports/*`, `/api/data-entry/*`, `/api/telegram/*`)
- [ ] Telegram bot runs in same process (background task), receives/sends messages
- [ ] Oracle pool initialized once, shared across all modules
- [ ] PaddleOCR loads in parallel with Oracle pool at startup (non-blocking)
- [ ] All logs appear in single console stream with module prefixes
- [ ] `/health` endpoint returns comprehensive status (Oracle, SQLite, Telegram bot)
- [ ] Frontend can access all functionality through unified backend
- [ ] Windows deployment requires only 1 NSSM service (not 3)
- [ ] All existing tests pass after migration
- [ ] Startup script simplified to single command (no parallel process management)
- [ ] Development mode: `uvicorn backend.main:app --reload` auto-reloads on code changes
- [ ] Module directories clearly separated (`modules/reports/`, `modules/data-entry/`, `modules/telegram/`)

## Out of Scope

**Explicitly NOT Included in This Transformation:**

- ❌ Frontend code changes (only configuration updates: vite.config.js, web.config)
- ❌ Business logic modifications (pure architectural refactoring)
- ❌ Database schema changes (no migrations needed)
- ❌ Authentication system changes (JWT logic stays same)
- ❌ API contract changes (same endpoints, just different base URL)
- ❌ Oracle database migration (stays on Oracle)
- ❌ SQLite schema changes (data-entry and telegram databases unchanged)
- ❌ UI/UX modifications (frontend functionality stays identical)
- ❌ Performance optimizations (beyond parallel initialization)
- ❌ New features (pure consolidation work)

## Risks and Mitigations

| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Import path issues after moving files | High | Medium | Use absolute imports with module prefixes (`from backend.modules.reports.services import ...`), update all imports systematically |
| Oracle pool initialization delays startup | Medium | Low | Use parallel initialization (Oracle + PaddleOCR concurrently), add timeout handling |
| Telegram bot blocks main thread | Medium | High | Run bot in background thread with `asyncio.create_task()` or `threading.Thread(daemon=True)` |
| PaddleOCR memory usage in same process | Low | Medium | Keep background initialization, monitor memory usage, add configuration to disable OCR if needed |
| Port 8000 conflicts with existing service | Low | Low | Make port configurable via environment variable, document port change |
| Frontend cache issues with URL changes | Medium | Low | Clear browser cache, document URL changes, use cache-busting headers |
| Windows service migration breaks existing deployment | Medium | High | Create new service name (`ROA2WEB-Unified`), test thoroughly before removing old services, document rollback procedure |
| Circular imports between modules | Low | Medium | Keep module dependencies one-way (reports/data-entry/telegram should not import each other), use shared code for common functionality |
| Environment variable conflicts | Low | Medium | Use namespaced prefixes (`REPORTS_*`, `DATA_ENTRY_*`, `TELEGRAM_*`), document all variables in `.env.example` |
| Lost logs during migration | Low | High | Keep old logs in archive, test logging thoroughly in dev environment first |

## Open Questions

1. **Telegram Bot Threading**: Should we use `asyncio.create_task()` (async) or `threading.Thread()` (sync) for Telegram bot polling?
   - **Recommendation**: Use `asyncio.create_task()` to stay in async context, avoid GIL issues

2. **Module Import Strategy**: Should modules import from each other, or only through shared code?
   - **Recommendation**: NO cross-module imports (reports ↔ data-entry ↔ telegram), only via `shared/`

3. **Health Check Depth**: How detailed should `/health` endpoint be (simple up/down vs full diagnostics)?
   - **Recommendation**: Return dict with per-module status, keep fast (<500ms)

4. **Backward Compatibility**: Should we support old URLs temporarily (redirect 8001→8000)?
   - **Recommendation**: NO - clean break, update frontend configuration once

5. **Development vs Production Config**: Should we have separate startup modes?
   - **Recommendation**: Use environment variables (`ENV=development|production`), single entry point

6. **Migration Strategy**: Big bang (all at once) or incremental (one module at a time)?
   - **Recommendation**: Big bang - easier to test, cleaner migration, no production deployment yet

7. **Logging Format**: Should we prefix all log messages with module name `[REPORTS]`, `[DATA-ENTRY]`, `[TELEGRAM]`?
   - **Recommendation**: YES - use Python logging with module-specific loggers

## Estimated Complexity

**HIGH** - Major architectural transformation requiring:

**Justification:**
- Moving and reorganizing ~50+ files across 3 backends
- Updating ~200+ import statements
- Merging 3 separate lifecycle managers into one
- Integrating Telegram bot from separate process into main app
- Testing all functionality after migration
- Updating deployment scripts and documentation
- Ensuring no regressions across all modules

**Estimated Effort:**
- File reorganization: 4-6 hours
- Import path updates: 3-4 hours
- Unified main.py + lifecycle: 3-4 hours
- Telegram bot integration: 2-3 hours
- Testing (manual + automated): 4-6 hours
- Frontend configuration updates: 1-2 hours
- Deployment script updates: 2-3 hours
- Documentation updates: 2-3 hours
- **Total: 21-31 hours (~3-4 working days)**

**Recommended Approach:**
1. Create new structure, copy files (don't delete old ones yet)
2. Update imports module by module
3. Test each module independently
4. Integrate Telegram bot last
5. Full integration testing
6. Update frontend configuration
7. Archive old backend directories after verification

## Implementation Notes

### Startup Sequence

```python
# backend/main.py - Startup lifespan
@asynccontextmanager
async def lifespan(app: FastAPI):
    """Unified startup/shutdown for all modules"""

    # 1. Initialize logging
    setup_logging()

    # 2. Parallel initialization (Oracle + PaddleOCR)
    await asyncio.gather(
        init_oracle_pool(),
        init_paddle_ocr(),  # Background initialization
        init_data_entry_db(),
        init_telegram_db()
    )

    # 3. Initialize module-specific components
    await init_reports_cache()

    # 4. Start Telegram bot (background task)
    telegram_task = asyncio.create_task(run_telegram_bot())

    logger.info("🚀 ROA2WEB Unified Backend started on port 8000")

    yield

    # Shutdown
    telegram_task.cancel()
    await close_all_resources()
```

### Router Registration Pattern

```python
# backend/main.py - Router registration
from modules.reports.routers import create_reports_router
from modules.data_entry.routers import create_data_entry_router
from modules.telegram.routers import create_telegram_router

# Register module routers with prefixes
app.include_router(create_reports_router(), prefix="/api/reports", tags=["reports"])
app.include_router(create_data_entry_router(), prefix="/api/data-entry", tags=["data-entry"])
app.include_router(create_telegram_router(), prefix="/api/telegram", tags=["telegram"])

# Register shared routers (no prefix)
app.include_router(auth_router, prefix="/api/auth", tags=["auth"])
app.include_router(companies_router, prefix="/api/companies", tags=["companies"])
app.include_router(calendar_router, prefix="/api/calendar", tags=["calendar"])
```

### Module Router Factory Pattern

```python
# backend/modules/reports/routers/__init__.py
from fastapi import APIRouter
from .invoices import router as invoices_router
from .dashboard import router as dashboard_router
from .treasury import router as treasury_router
# ... other routers

def create_reports_router() -> APIRouter:
    """Create and configure reports module router"""
    router = APIRouter()

    # Include all sub-routers (no prefix - already in main.py)
    router.include_router(invoices_router, prefix="/invoices")
    router.include_router(dashboard_router, prefix="/dashboard")
    router.include_router(treasury_router, prefix="/treasury")
    # ... other routers

    return router
```

### Critical Files List

**Top 10 Files That Will Be Affected:**

1. `/backend/main.py` - **NEW** - Single entry point, replaces 3 main.py files
2. `/backend/config.py` - **NEW** - Unified configuration
3. `/frontend/vite.config.js` - Update proxy configuration
4. `/frontend/public/web.config` - Update IIS rewrite rules
5. `reports-app/backend/app/main.py` - Move logic to unified main.py
6. `data-entry-app/backend/app/main.py` - Move logic to unified main.py
7. `reports-app/telegram-bot/app/main.py` - Move logic to unified main.py
8. `shared/database/oracle_pool.py` - Ensure singleton pattern for shared pool
9. `/deployment/windows/install-service.ps1` - Update to install single service
10. `/start-dev.sh` - Simplify to start single backend process

**Additional Critical Files:**
- All router files in `reports-app/backend/app/routers/` (~10 files)
- All router files in `data-entry-app/backend/app/routers/` (~4 files)
- Telegram bot handlers in `reports-app/telegram-bot/app/bot/handlers.py`
- Cache system in `reports-app/backend/app/cache/` (~9 files)
- OCR service in `data-entry-app/backend/app/services/ocr_service.py`
- Database models in `data-entry-app/backend/app/db/`

---

**Document Version**: 1.0
**Created**: 2025-12-24
**Author**: Auto-Build System
**Status**: Draft - Ready for Review