roa2web-service-auto/.auto-build/specs/ultrathin-monolith/spec.md

Feature: Ultrathin Monolith Backend Architecture

Overview

Transform ROA2WEB from 3 separate backend services (reports-app:8001, data-entry-app:8003, telegram-bot:8002) into a single unified backend process running on port 8000. This architectural transformation simplifies deployment, centralizes logs, and reduces operational complexity for solo developer workflow while maintaining clear module boundaries.

Problem Statement

Current Pain Points:

• Running 3 separate backend processes is complex to manage (start/stop/monitor)
• Windows deployment requires 3 NSSM services, significantly increasing deployment time
• Debugging is difficult with fragmented logs across multiple processes
• Resource overhead of multiple Python processes and connection pools
• Frontend already unified into single SPA, but backend remains fragmented

Who Benefits:

• Solo developer: Easier development workflow, simplified debugging
• Windows deployment: Single service instead of 3 (faster deployment)
• System resources: Single Oracle pool, single process overhead

User Stories

• As a developer, I want to start the entire backend with a single command (python main.py) so that I can begin working faster
• As a developer, I want centralized logs in a single stream so that I can debug issues across all modules more easily
• As a deployer, I want to install a single Windows service so that deployment takes less time
• As a system administrator, I want a single backend process so that resource usage is more efficient
• As a developer, I want to keep separate module directories (reports/, data-entry/, telegram-bot/) so that code organization remains clear

Functional Requirements

Core Requirements

1. Single Entry Point

   • One main.py file at project root (/backend/main.py)
   • Single FastAPI application instance
   • Single uvicorn process running on port 8000

2. Module Prefix Routing

   • Reports endpoints: /api/reports/* (e.g., /api/reports/invoices, /api/reports/dashboard)
   • Data Entry endpoints: /api/data-entry/* (e.g., /api/data-entry/receipts, /api/data-entry/ocr)
   • Telegram endpoints: /api/telegram/* (e.g., /api/telegram/auth/verify-user)
   • Shared endpoints: /api/auth/*, /api/companies, /api/calendar (no prefix change)

3. Integrated Telegram Bot

   • Telegram bot runs in same process (background thread/task)
   • Internal API endpoints accessible via /api/telegram/*
   • Shared database access (no separate SQLite for bot)

4. Unified Startup/Shutdown

   • Single lifespan manager initializing all modules
   • Parallel initialization where possible (Oracle pool + PaddleOCR)
   • Graceful shutdown for all components
   • Centralized logging configuration

5. Module Directory Structure

   • Keep existing directories: backend/modules/reports/, backend/modules/data-entry/, backend/modules/telegram/
   • Each module retains its own routers, services, models
   • Shared code remains in shared/ (auth, database, routes)

Secondary Requirements

1. Environment Variable Consolidation

   • Single .env file at backend/.env
   • Merge variables from all 3 backends
   • Namespace conflicts resolved (e.g., REPORTS_*, DATA_ENTRY_* prefixes)

2. Single Virtual Environment

   • Merged requirements.txt with all dependencies
   • De-duplicated dependencies (single version of FastAPI, etc.)
   • Optional dependency groups for OCR/Telegram

3. Health Check Aggregation

   • Single /health endpoint reporting status of all modules
   • Check Oracle connection, SQLite database, Telegram bot status
   • Return comprehensive health information

4. Code Cleanup

   • Remove duplicate auth router implementations (use shared router factory)
   • Remove duplicate company/calendar router implementations
   • Consolidate logging configuration
   • Remove unused imports and code

Technical Requirements

Files to Create

File	Purpose
/backend/main.py	Single unified entry point - creates FastAPI app, registers all module routers, manages lifecycle
/backend/config.py	Unified configuration management (merges settings from all apps)
/backend/requirements.txt	Consolidated dependencies from all 3 backends
/backend/.env.example	Merged environment variables template
/backend/modules/__init__.py	Module package marker
/backend/modules/reports/__init__.py	Reports module package
/backend/modules/data-entry/__init__.py	Data entry module package
/backend/modules/telegram/__init__.py	Telegram module package

Files to Modify

File	Changes
reports-app/backend/app/routers/*.py	Move to backend/modules/reports/routers/
reports-app/backend/app/services/*.py	Move to backend/modules/reports/services/
reports-app/backend/app/models/*.py	Move to backend/modules/reports/models/
reports-app/backend/app/cache/**/*.py	Move to backend/modules/reports/cache/ (reports-specific caching)
data-entry-app/backend/app/routers/*.py	Move to backend/modules/data-entry/routers/
data-entry-app/backend/app/services/*.py	Move to backend/modules/data-entry/services/
data-entry-app/backend/app/db/**/*.py	Move to backend/modules/data-entry/db/
data-entry-app/backend/migrations/	Move to backend/modules/data-entry/migrations/
reports-app/telegram-bot/app/bot/*.py	Move to backend/modules/telegram/bot/
reports-app/telegram-bot/app/db/*.py	Move to backend/modules/telegram/db/
reports-app/telegram-bot/app/internal_api.py	Integrate into main FastAPI app as router
All router files	Update imports to reflect new paths
vite.config.js	Update proxy configuration to single backend (port 8000)
public/web.config (IIS)	Update proxy rules to single backend (port 8000)

New Directory Structure

roa2web/
├── backend/                         # UNIFIED BACKEND
│   ├── main.py                      # Single entry point
│   ├── config.py                    # Unified configuration
│   ├── requirements.txt             # Merged dependencies
│   ├── .env                         # Merged environment variables
│   ├── .env.example                 # Environment template
│   ├── venv/                        # Single virtual environment
│   │
│   ├── modules/                     # Module-specific code
│   │   ├── __init__.py
│   │   │
│   │   ├── reports/                 # Reports module (from reports-app)
│   │   │   ├── __init__.py
│   │   │   ├── routers/             # API endpoints (invoices, dashboard, treasury, etc.)
│   │   │   ├── services/            # Business logic
│   │   │   ├── models/              # Pydantic models
│   │   │   ├── schemas/             # Response schemas
│   │   │   └── cache/               # Caching system (L1+L2)
│   │   │
│   │   ├── data-entry/              # Data entry module (from data-entry-app)
│   │   │   ├── __init__.py
│   │   │   ├── routers/             # Receipts, OCR, nomenclature routers
│   │   │   ├── services/            # Business logic + OCR service
│   │   │   ├── db/                  # SQLModel models + CRUD
│   │   │   ├── schemas/             # Pydantic schemas
│   │   │   └── migrations/          # Alembic migrations
│   │   │
│   │   └── telegram/                # Telegram bot module (from telegram-bot)
│   │       ├── __init__.py
│   │       ├── bot/                 # Command handlers, formatters
│   │       ├── db/                  # Bot-specific database (auth codes, sessions)
│   │       └── routers/             # Internal API endpoints (auth code management)
│   │
│   └── data/                        # Data files
│       ├── cache/                   # SQLite cache files (reports module)
│       ├── receipts/                # SQLite DB + uploads (data-entry module)
│       └── telegram/                # SQLite DB for bot (telegram module)
│
├── shared/                          # Shared components (unchanged)
│   ├── auth/                        # JWT authentication
│   ├── database/                    # Oracle pool
│   ├── routes/                      # Shared router factories
│   └── frontend/                    # Shared Vue components
│
├── frontend/                        # Unified frontend (unchanged structure)
│   ├── src/
│   │   ├── modules/
│   │   │   ├── reports/
│   │   │   └── data-entry/
│   │   └── shared/
│   └── vite.config.js               # UPDATE: Single proxy to localhost:8000
│
├── deployment/                      # Deployment scripts
│   └── windows/                     # Windows deployment
│       └── install-service.ps1      # UPDATE: Single NSSM service
│
└── docs/                            # Documentation

Dependencies Consolidation

Common Dependencies (deduplicated):

fastapi>=0.109.0
uvicorn[standard]>=0.27.0
pydantic>=2.5.3
python-dotenv>=1.0.0
PyJWT>=2.8.0
python-jose[cryptography]>=3.3.0
oracledb>=2.0.1
aiosqlite>=0.19.0
httpx>=0.27.0
python-multipart>=0.0.6

Reports-Specific:

openpyxl>=3.1.0              # Excel export
fpdf2>=2.7.0                 # PDF generation
python-dateutil>=2.8.2       # Date utilities

Data Entry-Specific:

sqlmodel>=0.0.14             # ORM
alembic>=1.13.1              # Migrations
aiofiles>=23.2.1             # Async file handling
Pillow>=10.2.0               # Image processing
paddleocr>=2.7.0             # OCR engine
paddlepaddle>=2.5.0          # PaddleOCR backend
opencv-python>=4.8.0         # Image processing
pytesseract>=0.3.10          # Tesseract OCR
pdf2image>=1.16.0            # PDF to image conversion
numpy>=1.24.0                # Array operations

Telegram-Specific:

python-telegram-bot>=20.7    # Telegram bot SDK
aiosmtplib>=3.0.0            # Email (2FA)
sentry-sdk>=1.40.0           # Monitoring (optional)

Database Changes

No schema changes - databases remain separate:

• Oracle: Shared via oracle_pool (reports + data-entry nomenclatures + auth)
• SQLite (receipts.db): Data entry module receipts data
• SQLite (cache.db): Reports module caching
• SQLite (telegram.db): Telegram bot auth codes and sessions

Database Access Patterns:

• Oracle Pool: Single shared instance initialized at startup, used by all modules
• SQLite databases: Separate per module, stored in backend/data/

API Changes

URL Migration (frontend must update):

Old URL	New URL	Module
http://localhost:8001/api/invoices	http://localhost:8000/api/reports/invoices	Reports
http://localhost:8001/api/dashboard	http://localhost:8000/api/reports/dashboard	Reports
http://localhost:8001/api/treasury	http://localhost:8000/api/reports/treasury	Reports
http://localhost:8001/api/trial-balance	http://localhost:8000/api/reports/trial-balance	Reports
http://localhost:8001/api/cache/*	http://localhost:8000/api/reports/cache/*	Reports
http://localhost:8003/api/receipts	http://localhost:8000/api/data-entry/receipts	Data Entry
http://localhost:8003/api/ocr	http://localhost:8000/api/data-entry/ocr	Data Entry
http://localhost:8003/api/nomenclature	http://localhost:8000/api/data-entry/nomenclature	Data Entry
http://localhost:8002/api/telegram/*	http://localhost:8000/api/telegram/*	Telegram
http://localhost:8001/api/auth/login	http://localhost:8000/api/auth/login	Shared (no change)
http://localhost:8001/api/companies	http://localhost:8000/api/companies	Shared (no change)
http://localhost:8001/api/calendar	http://localhost:8000/api/calendar	Shared (no change)

Frontend Configuration Updates:

// vite.config.js - BEFORE (3 proxies)
proxy: {
  '/api/reports': { target: 'http://localhost:8001', rewrite: (path) => path.replace(/^\/api\/reports/, '/api') },
  '/api/data-entry': { target: 'http://localhost:8003', rewrite: (path) => path.replace(/^\/api\/data-entry/, '/api') },
  '/api/telegram': { target: 'http://localhost:8002', rewrite: (path) => path.replace(/^\/api\/telegram/, '/api') }
}

// vite.config.js - AFTER (1 proxy)
proxy: {
  '/api': { target: 'http://localhost:8000', changeOrigin: true }
}

<!-- public/web.config - BEFORE (3 rules) -->
<rule name="Proxy Reports API" stopProcessing="true">
  <match url="^api/reports/(.*)" />
  <action type="Rewrite" url="http://localhost:8001/api/{R:1}" />
</rule>
<rule name="Proxy Data Entry API" stopProcessing="true">
  <match url="^api/data-entry/(.*)" />
  <action type="Rewrite" url="http://localhost:8003/api/{R:1}" />
</rule>
<rule name="Proxy Telegram API" stopProcessing="true">
  <match url="^api/telegram/(.*)" />
  <action type="Rewrite" url="http://localhost:8002/api/{R:1}" />
</rule>

<!-- public/web.config - AFTER (1 rule) -->
<rule name="Proxy Backend API" stopProcessing="true">
  <match url="^api/(.*)" />
  <action type="Rewrite" url="http://localhost:8000/api/{R:1}" />
</rule>

Design Decisions

Approach: Module-Based Monolith with Clear Boundaries

Why This Approach:

1. Maintains Module Isolation: Each module (reports, data-entry, telegram) keeps its own directory structure
2. Simplifies Deployment: Single process, single service, single log stream
3. Preserves Code Organization: Module boundaries remain clear in file structure
4. Enables Gradual Migration: Can move one module at a time
5. Reduces Complexity: Solo developer doesn't need microservices orchestration

Key Architectural Patterns:

• Single FastAPI App: One application instance with multiple routers registered with prefixes
• Module Routers: Each module exports a function that returns its configured router
• Parallel Initialization: Oracle pool and PaddleOCR load concurrently at startup
• Unified Logging: Single logging configuration with module-specific loggers
• Shared Singleton: Oracle pool shared across all modules

Alternatives Considered

Alternative 1: Keep Microservices Architecture

• Rejected: Adds complexity without benefits for solo developer
• Rejected: Windows deployment remains slow (3 services)
• Rejected: Debugging remains difficult (fragmented logs)

Alternative 2: Merge Everything into Flat Structure

• Rejected: Loses module boundaries, code becomes harder to navigate
• Rejected: Difficult to understand which code belongs to which feature

Alternative 3: Use Docker Compose for Orchestration

• Rejected: Adds Docker overhead without solving core issues
• Rejected: Windows deployment still needs Docker Desktop or manual setup
• Rejected: Doesn't simplify solo developer workflow

Acceptance Criteria

• Single python backend/main.py command starts entire backend
• All endpoints accessible on port 8000 with module prefixes (/api/reports/*, /api/data-entry/*, /api/telegram/*)
• Telegram bot runs in same process (background task), receives/sends messages
• Oracle pool initialized once, shared across all modules
• PaddleOCR loads in parallel with Oracle pool at startup (non-blocking)
• All logs appear in single console stream with module prefixes
• /health endpoint returns comprehensive status (Oracle, SQLite, Telegram bot)
• Frontend can access all functionality through unified backend
• Windows deployment requires only 1 NSSM service (not 3)
• All existing tests pass after migration
• Startup script simplified to single command (no parallel process management)
• Development mode: uvicorn backend.main:app --reload auto-reloads on code changes
• Module directories clearly separated (modules/reports/, modules/data-entry/, modules/telegram/)

Out of Scope

Explicitly NOT Included in This Transformation:

• ❌ Frontend code changes (only configuration updates: vite.config.js, web.config)
• ❌ Business logic modifications (pure architectural refactoring)
• ❌ Database schema changes (no migrations needed)
• ❌ Authentication system changes (JWT logic stays same)
• ❌ API contract changes (same endpoints, just different base URL)
• ❌ Oracle database migration (stays on Oracle)
• ❌ SQLite schema changes (data-entry and telegram databases unchanged)
• ❌ UI/UX modifications (frontend functionality stays identical)
• ❌ Performance optimizations (beyond parallel initialization)
• ❌ New features (pure consolidation work)

Risks and Mitigations

Risk	Likelihood	Impact	Mitigation
Import path issues after moving files	High	Medium	Use absolute imports with module prefixes (from backend.modules.reports.services import ...), update all imports systematically
Oracle pool initialization delays startup	Medium	Low	Use parallel initialization (Oracle + PaddleOCR concurrently), add timeout handling
Telegram bot blocks main thread	Medium	High	Run bot in background thread with asyncio.create_task() or threading.Thread(daemon=True)
PaddleOCR memory usage in same process	Low	Medium	Keep background initialization, monitor memory usage, add configuration to disable OCR if needed
Port 8000 conflicts with existing service	Low	Low	Make port configurable via environment variable, document port change
Frontend cache issues with URL changes	Medium	Low	Clear browser cache, document URL changes, use cache-busting headers
Windows service migration breaks existing deployment	Medium	High	Create new service name (ROA2WEB-Unified), test thoroughly before removing old services, document rollback procedure
Circular imports between modules	Low	Medium	Keep module dependencies one-way (reports/data-entry/telegram should not import each other), use shared code for common functionality
Environment variable conflicts	Low	Medium	Use namespaced prefixes (REPORTS_*, DATA_ENTRY_*, TELEGRAM_*), document all variables in .env.example
Lost logs during migration	Low	High	Keep old logs in archive, test logging thoroughly in dev environment first

Open Questions

1. Telegram Bot Threading: Should we use asyncio.create_task() (async) or threading.Thread() (sync) for Telegram bot polling?

   • Recommendation: Use asyncio.create_task() to stay in async context, avoid GIL issues

2. Module Import Strategy: Should modules import from each other, or only through shared code?

   • Recommendation: NO cross-module imports (reports ↔ data-entry ↔ telegram), only via shared/

3. Health Check Depth: How detailed should /health endpoint be (simple up/down vs full diagnostics)?

   • Recommendation: Return dict with per-module status, keep fast (<500ms)

4. Backward Compatibility: Should we support old URLs temporarily (redirect 8001→8000)?

   • Recommendation: NO - clean break, update frontend configuration once

5. Development vs Production Config: Should we have separate startup modes?

   • Recommendation: Use environment variables (ENV=development|production), single entry point

6. Migration Strategy: Big bang (all at once) or incremental (one module at a time)?

   • Recommendation: Big bang - easier to test, cleaner migration, no production deployment yet

7. Logging Format: Should we prefix all log messages with module name [REPORTS], [DATA-ENTRY], [TELEGRAM]?

   • Recommendation: YES - use Python logging with module-specific loggers

Estimated Complexity

HIGH - Major architectural transformation requiring:

Justification:

• Moving and reorganizing ~50+ files across 3 backends
• Updating ~200+ import statements
• Merging 3 separate lifecycle managers into one
• Integrating Telegram bot from separate process into main app
• Testing all functionality after migration
• Updating deployment scripts and documentation
• Ensuring no regressions across all modules

Estimated Effort:

• File reorganization: 4-6 hours
• Import path updates: 3-4 hours
• Unified main.py + lifecycle: 3-4 hours
• Telegram bot integration: 2-3 hours
• Testing (manual + automated): 4-6 hours
• Frontend configuration updates: 1-2 hours
• Deployment script updates: 2-3 hours
• Documentation updates: 2-3 hours
• Total: 21-31 hours (~3-4 working days)

Recommended Approach:

1. Create new structure, copy files (don't delete old ones yet)
2. Update imports module by module
3. Test each module independently
4. Integrate Telegram bot last
5. Full integration testing
6. Update frontend configuration
7. Archive old backend directories after verification

Implementation Notes

Startup Sequence

# backend/main.py - Startup lifespan
@asynccontextmanager
async def lifespan(app: FastAPI):
    """Unified startup/shutdown for all modules"""

    # 1. Initialize logging
    setup_logging()

    # 2. Parallel initialization (Oracle + PaddleOCR)
    await asyncio.gather(
        init_oracle_pool(),
        init_paddle_ocr(),  # Background initialization
        init_data_entry_db(),
        init_telegram_db()
    )

    # 3. Initialize module-specific components
    await init_reports_cache()

    # 4. Start Telegram bot (background task)
    telegram_task = asyncio.create_task(run_telegram_bot())

    logger.info("🚀 ROA2WEB Unified Backend started on port 8000")

    yield

    # Shutdown
    telegram_task.cancel()
    await close_all_resources()

Router Registration Pattern

# backend/main.py - Router registration
from modules.reports.routers import create_reports_router
from modules.data_entry.routers import create_data_entry_router
from modules.telegram.routers import create_telegram_router

# Register module routers with prefixes
app.include_router(create_reports_router(), prefix="/api/reports", tags=["reports"])
app.include_router(create_data_entry_router(), prefix="/api/data-entry", tags=["data-entry"])
app.include_router(create_telegram_router(), prefix="/api/telegram", tags=["telegram"])

# Register shared routers (no prefix)
app.include_router(auth_router, prefix="/api/auth", tags=["auth"])
app.include_router(companies_router, prefix="/api/companies", tags=["companies"])
app.include_router(calendar_router, prefix="/api/calendar", tags=["calendar"])

Module Router Factory Pattern

# backend/modules/reports/routers/__init__.py
from fastapi import APIRouter
from .invoices import router as invoices_router
from .dashboard import router as dashboard_router
from .treasury import router as treasury_router
# ... other routers

def create_reports_router() -> APIRouter:
    """Create and configure reports module router"""
    router = APIRouter()

    # Include all sub-routers (no prefix - already in main.py)
    router.include_router(invoices_router, prefix="/invoices")
    router.include_router(dashboard_router, prefix="/dashboard")
    router.include_router(treasury_router, prefix="/treasury")
    # ... other routers

    return router

Critical Files List

Top 10 Files That Will Be Affected:

1. /backend/main.py - NEW - Single entry point, replaces 3 main.py files
2. /backend/config.py - NEW - Unified configuration
3. /frontend/vite.config.js - Update proxy configuration
4. /frontend/public/web.config - Update IIS rewrite rules
5. reports-app/backend/app/main.py - Move logic to unified main.py
6. data-entry-app/backend/app/main.py - Move logic to unified main.py
7. reports-app/telegram-bot/app/main.py - Move logic to unified main.py
8. shared/database/oracle_pool.py - Ensure singleton pattern for shared pool
9. /deployment/windows/install-service.ps1 - Update to install single service
10. /start-dev.sh - Simplify to start single backend process

Additional Critical Files:

• All router files in reports-app/backend/app/routers/ (~10 files)
• All router files in data-entry-app/backend/app/routers/ (~4 files)
• Telegram bot handlers in reports-app/telegram-bot/app/bot/handlers.py
• Cache system in reports-app/backend/app/cache/ (~9 files)
• OCR service in data-entry-app/backend/app/services/ocr_service.py
• Database models in data-entry-app/backend/app/db/

---

Document Version: 1.0 Created: 2025-12-24 Author: Auto-Build System Status: Draft - Ready for Review