feat: Migrate to ultrathin monolith architecture
Consolidate 3 separate applications (reports-app, data-entry-app, telegram-bot) into a unified
architecture with single backend and frontend:
Backend Changes:
- Unified FastAPI backend at backend/ with modular structure
- Modules: reports, data_entry, telegram in backend/modules/
- Centralized config.py and main.py with all routers registered
- Single worker mode (--workers 1) for Telegram bot compatibility
- Shared Oracle connection pool and JWT authentication
- Unified requirements.txt and environment configuration
Frontend Changes:
- Single Vue.js SPA with module-based routing
- Unified frontend at src/ with modules in src/modules/{reports,data-entry}/
- Shared components and stores in src/shared/
- Error boundaries for module isolation
- Dual API proxy in Vite for module communication
Infrastructure:
- New unified startup scripts: start-prod.sh, start-test.sh, start-backend.sh
- Environment templates: .env.dev.example, .env.test.example, .env.prod.example
- Updated deployment scripts for Windows IIS
- Simplified SSH tunnel management
Documentation:
- Comprehensive CLAUDE.md with architecture overview
- Module-specific docs in docs/{data-entry,telegram}/
- Architecture decision records in docs/ARCHITECTURE-DECISIONS.md
- Deployment guides consolidated in deployment/windows/docs/
This migration reduces complexity, improves maintainability, and enables easier
deployment while maintaining all existing functionality.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
176
docs/ARCHITECTURE-DECISIONS.md
Normal file
176
docs/ARCHITECTURE-DECISIONS.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# Architecture Decision Records (ADR)
|
||||
|
||||
This document records important architectural decisions made during ROA2WEB development.
|
||||
|
||||
---
|
||||
|
||||
## ADR-001: Ultrathin Monolith Architecture (2025-12-24)
|
||||
|
||||
**Status:** Accepted
|
||||
|
||||
**Context:**
|
||||
Originally designed as separate microservices (Reports, Data Entry, Telegram Bot as independent services).
|
||||
|
||||
**Decision:**
|
||||
Consolidated into **ultrathin monolith** - single FastAPI application with modular structure.
|
||||
|
||||
**Consequences:**
|
||||
- ✅ Single Windows service to manage
|
||||
- ✅ Shared database connection pool
|
||||
- ✅ Simpler deployment
|
||||
- ✅ Lower memory usage
|
||||
- ✅ Easier debugging
|
||||
- ⚠️ Requires careful module isolation
|
||||
|
||||
**Implementation:**
|
||||
- All modules under `backend/modules/`
|
||||
- Single `ROA2WEB-Backend` service
|
||||
- Module enable/disable via `.env` flags
|
||||
|
||||
---
|
||||
|
||||
## ADR-002: Single Worker Configuration (2025-12-29)
|
||||
|
||||
**Status:** Accepted
|
||||
|
||||
**Context:**
|
||||
Telegram bot integration causes conflicts when multiple uvicorn workers run simultaneously. Each worker spawns its own bot instance, but Telegram API allows only ONE bot to poll for updates.
|
||||
|
||||
**Problem:**
|
||||
```
|
||||
telegram.error.Conflict: terminated by other getUpdates request
|
||||
```
|
||||
|
||||
**Decision:**
|
||||
**Always use `--workers 1` for the unified backend service.**
|
||||
|
||||
**Rationale:**
|
||||
1. **Telegram Bot Requirement:** Single bot instance only
|
||||
2. **SQLite Cache:** Multiple workers cause locking conflicts
|
||||
3. **Performance:** Async I/O means single worker handles 100+ concurrent requests
|
||||
4. **Resources:** Lower memory (~400 MB vs ~1.6 GB with 4 workers)
|
||||
|
||||
**Consequences:**
|
||||
- ✅ Telegram bot works perfectly
|
||||
- ✅ No SQLite cache conflicts
|
||||
- ✅ Cache stats endpoint works (no 502 errors)
|
||||
- ✅ Lower memory usage
|
||||
- ✅ Same performance (I/O bound, not CPU bound)
|
||||
- ⚠️ Cannot scale horizontally with multiple processes
|
||||
|
||||
**Implementation:**
|
||||
- Configured in NSSM: `--workers 1`
|
||||
- NOT configurable in `.env`
|
||||
- Documented in `TELEGRAM-BOT-DEPLOYMENT.md`
|
||||
|
||||
**Alternatives Considered:**
|
||||
- Separate bot service: More complexity, no significant benefit
|
||||
- Redis pub/sub for bot: Over-engineering for current scale
|
||||
|
||||
**Performance Characteristics:**
|
||||
- Concurrent users: 100+
|
||||
- Requests per minute: 1000+
|
||||
- Response time: 50-200ms (DB query time)
|
||||
- Adequate for: 50-100 concurrent users
|
||||
|
||||
---
|
||||
|
||||
## ADR-003: IIS Sub-Application Deployment (2025-12-29)
|
||||
|
||||
**Status:** Accepted
|
||||
|
||||
**Context:**
|
||||
Application deployed at `/roa2web` path on IIS, not at root.
|
||||
|
||||
**Decision:**
|
||||
Use Vite `base: '/roa2web/'` and axios `baseURL: import.meta.env.BASE_URL + 'api'` for all frontend API calls.
|
||||
|
||||
**Consequences:**
|
||||
- ✅ Works correctly on production sub-path
|
||||
- ✅ Works correctly on development root path
|
||||
- ⚠️ All axios instances must use BASE_URL
|
||||
- ⚠️ Router must use base path
|
||||
|
||||
**Implementation:**
|
||||
- `vite.config.js`: `base: process.env.NODE_ENV === 'production' ? '/roa2web/' : '/'`
|
||||
- All `axios.create()`: `baseURL: import.meta.env.BASE_URL + 'api/...'`
|
||||
- IIS `web.config`: Reverse proxy rules for `/api/*` → `localhost:8000`
|
||||
|
||||
---
|
||||
|
||||
## ADR-004: Pydantic v2 Migration (2025-12-29)
|
||||
|
||||
**Status:** In Progress
|
||||
|
||||
**Context:**
|
||||
Pydantic v2 changed `schema_extra` → `json_schema_extra`.
|
||||
|
||||
**Decision:**
|
||||
Update all models incrementally to use `json_schema_extra`.
|
||||
|
||||
**Implementation:**
|
||||
- ✅ `backend/modules/reports/models/trial_balance.py` updated
|
||||
- ✅ `backend/modules/data_entry/schemas/ocr.py` already correct
|
||||
- ⚠️ Warning eliminated in logs
|
||||
|
||||
---
|
||||
|
||||
## ADR-005: Web.config as Static File (2025-12-29)
|
||||
|
||||
**Status:** Accepted
|
||||
|
||||
**Context:**
|
||||
IIS requires `web.config` for URL rewrite rules, but it wasn't consistently deployed.
|
||||
|
||||
**Decision:**
|
||||
1. Store canonical `web.config` in `deployment/windows/config/`
|
||||
2. Create `Create-WebConfig.ps1` script for manual creation
|
||||
3. Include in deployment package
|
||||
|
||||
**Consequences:**
|
||||
- ✅ Consistent configuration across deployments
|
||||
- ✅ Easy to recreate if deleted
|
||||
- ✅ Version controlled
|
||||
|
||||
**Implementation:**
|
||||
- Canonical: `deployment/windows/config/web.config`
|
||||
- Deployed to: `C:\inetpub\wwwroot\roa2web\frontend\web.config`
|
||||
- Creation script: `Create-WebConfig.ps1`
|
||||
|
||||
---
|
||||
|
||||
## Future Decisions to Document
|
||||
|
||||
### Pending Considerations
|
||||
|
||||
1. **Horizontal Scaling:**
|
||||
- If needed: Load balancer + multiple app instances
|
||||
- Alternative: Separate bot service to allow multiple workers
|
||||
- Current decision: Single worker sufficient
|
||||
|
||||
2. **Cache Strategy:**
|
||||
- Current: SQLite L2 cache + memory L1
|
||||
- Alternative: Redis for distributed cache
|
||||
- Decision: Keep SQLite for simplicity
|
||||
|
||||
3. **Database Connection Pool:**
|
||||
- Current: Shared pool across all modules
|
||||
- Sizing: 5-10 connections
|
||||
- Per-module pools: Not needed yet
|
||||
|
||||
---
|
||||
|
||||
## Decision Process
|
||||
|
||||
When making architectural decisions:
|
||||
|
||||
1. **Document the context** - Why is this decision needed?
|
||||
2. **List alternatives** - What other options exist?
|
||||
3. **Explain rationale** - Why this option over others?
|
||||
4. **Note consequences** - What are the trade-offs?
|
||||
5. **Implementation details** - How is this implemented?
|
||||
6. **Reference documentation** - Where to find more details?
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-12-29
|
||||
@@ -1,482 +0,0 @@
|
||||
# 📊 ROA2WEB - SCHEMĂ GRAFICĂ ARHITECTURĂ
|
||||
|
||||
Această schemă prezintă arhitectura completă a aplicației ROA2WEB, incluzând frontend-ul Vue.js, backend-ul FastAPI, middleware-ul de autentificare și conexiunea la baza de date Oracle.
|
||||
|
||||
## 🏗️ **ARHITECTURA GENERALĂ**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🌐 CLIENT │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ HTTP Requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🖥️ FRONTEND │
|
||||
│ Vue.js 3 + PrimeVue + Vite │
|
||||
│ Port: 5173 (dev) / 3000 (prod) │
|
||||
│ │
|
||||
│ 📁 Components: 📦 Stores (Pinia): │
|
||||
│ • LoginView.vue • auth.js (JWT tokens) │
|
||||
│ • DashboardView.vue • companies.js │
|
||||
│ • InvoicesView.vue • dashboard.js │
|
||||
│ • BankCashRegisterView.vue • invoices.js │
|
||||
│ • treasury.js │
|
||||
│ 🔧 Services: │
|
||||
│ • api.js (Axios HTTP client) │
|
||||
│ • JWT token management │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ API Calls (axios)
|
||||
│ Authorization: Bearer <JWT>
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🚀 BACKEND API │
|
||||
│ FastAPI + Uvicorn │
|
||||
│ Port: 8000 │
|
||||
│ │
|
||||
│ 🛡️ MIDDLEWARE LAYER: │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 1. CORSMiddleware (Frontend communication) │ │
|
||||
│ │ 2. AuthenticationMiddleware (JWT validation) │ │
|
||||
│ │ • Token extraction from Authorization header │ │
|
||||
│ │ • JWT verification & user data injection │ │
|
||||
│ │ • Rate limiting (5 req/5min per IP) │ │
|
||||
│ │ • Security headers injection │ │
|
||||
│ │ • Excluded paths: ["/", "/docs", "/health", "/api/auth/login"] │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ 🛤️ API ROUTES: │
|
||||
│ • /api/auth/login (POST) - User authentication │
|
||||
│ • /api/companies (GET) - Company list │
|
||||
│ • /api/dashboard (GET) - Dashboard data │
|
||||
│ • /api/invoices (GET) - Invoice reports │
|
||||
│ • /api/treasury (GET) - Treasury/Bank data │
|
||||
│ • /api/trial-balance (GET) - Trial Balance (Balanță de Verificare) │
|
||||
│ • /health (GET) - Health check │
|
||||
│ │
|
||||
│ 📊 SERVICES (with caching): │
|
||||
│ • invoice_service.py (@cached: invoices, schema) │
|
||||
│ • dashboard_service.py (@cached: dashboard_summary, trends, etc.) │
|
||||
│ • treasury_service.py (@cached: treasury, schema) │
|
||||
│ │
|
||||
│ ⚡ CACHE LAYER (Two-Tier): │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ L1 (Memory): Fast in-memory cache (LRU, max 1000 entries) │ │
|
||||
│ │ L2 (SQLite): Persistent cache database (./cache_data/) │ │
|
||||
│ │ │ │
|
||||
│ │ Features: │ │
|
||||
│ │ • Automatic TTL per cache type (schema: 24h, invoices: 10min, etc.) │ │
|
||||
│ │ • Performance tracking & benchmarking │ │
|
||||
│ │ • Per-user cache enable/disable │ │
|
||||
│ │ • Event-based invalidation (optional) │ │
|
||||
│ │ • Cache hit/miss metrics in response headers │ │
|
||||
│ │ │ │
|
||||
│ │ Usage: @cached(cache_type='...', key_params=['company', 'username']) │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ Database Queries (on cache miss)
|
||||
│ SSH Tunnel Required
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🔐 SSH TUNNEL LAYER │
|
||||
│ ./ssh_tunnel.sh (Local port forwarding) │
|
||||
│ Local: localhost:1526 ➜ Remote: oracle_server:1521 │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ Encrypted connection
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🏛️ ORACLE DATABASE │
|
||||
│ Schema: CONTAFIN_ORACLE │
|
||||
│ Port: 1521 (remote) / 1526 (local via tunnel) │
|
||||
│ │
|
||||
│ 📋 Main Tables/Views: │
|
||||
│ • UTILIZATORI (Users) │
|
||||
│ • V_NOM_FIRME (Companies) │
|
||||
│ • VDEF_UTIL_FIRME (User-Company relations) │
|
||||
│ • Financial data tables (invoices, payments, etc.) │
|
||||
│ │
|
||||
│ 🔧 Stored Procedures: │
|
||||
│ • pack_drepturi.verificautilizator (Authentication) │
|
||||
└─────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## ⚡ **ARHITECTURA CACHE (TWO-TIER L1 + L2)**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🔥 CACHE LAYER ARCHITECTURE │
|
||||
└─────────────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
API Request (service method decorated with @cached)
|
||||
↓
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 1. Check if Cache Enabled (Global + Per-User) │
|
||||
│ • Global setting: CACHE_ENABLED=True/False │
|
||||
│ • Per-user setting: SQLite user_settings table │
|
||||
└───┬─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓ (if disabled: query Oracle directly)
|
||||
┌───▼─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 2. Generate Cache Key │
|
||||
│ • Pattern: {cache_type}:{param1}:{param2}:... │
|
||||
│ • Example: "invoices:123:user1:2024-01" │
|
||||
└───┬─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
┌───▼─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 3. Try L1 (Memory Cache) │
|
||||
│ • Python dict with TTL │
|
||||
│ • LRU eviction (max 1000 entries) │
|
||||
│ • ⚡ Ultra-fast (microseconds) │
|
||||
│ └─→ HIT? Return value + track performance (L1) │
|
||||
└───┬─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓ (if MISS)
|
||||
┌───▼─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 4. Try L2 (SQLite Cache) │
|
||||
│ • Persistent database (./cache_data/roa2web_cache.db) │
|
||||
│ • Indexed by key, company_id, cache_type │
|
||||
│ • 🗄️ Slower than L1 but faster than Oracle │
|
||||
│ └─→ HIT? Populate L1 + return value + track performance (L2) │
|
||||
└───┬─────────────────────────────────────────────────────────────────────────────┘
|
||||
↓ (if MISS)
|
||||
┌───▼─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 5. CACHE MISS - Query Oracle │
|
||||
│ • Execute actual database query │
|
||||
│ • Measure execution time (benchmark) │
|
||||
│ • Store result in L1 + L2 │
|
||||
│ • Track performance (cache miss) │
|
||||
└─────────────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
📊 CACHE TYPES & TTL (Time To Live):
|
||||
|
||||
• schema: 24 hours (CACHE_TTL_SCHEMA=86400)
|
||||
• companies: 30 min (CACHE_TTL_COMPANIES=1800)
|
||||
• dashboard_summary: 30 min (CACHE_TTL_DASHBOARD_SUMMARY=1800)
|
||||
• dashboard_trends: 30 min (CACHE_TTL_DASHBOARD_TRENDS=1800)
|
||||
• invoices: 10 min (CACHE_TTL_INVOICES=600)
|
||||
• invoices_summary: 15 min (CACHE_TTL_INVOICES_SUMMARY=900)
|
||||
• treasury: 10 min (CACHE_TTL_TREASURY=600)
|
||||
• trial_balance: 10 min (CACHE_TTL_TRIAL_BALANCE=600)
|
||||
|
||||
🔧 CACHE MANAGEMENT ENDPOINTS:
|
||||
|
||||
• GET /api/cache/stats - Cache statistics (hits, misses, performance)
|
||||
• POST /api/cache/invalidate - Invalidate cache (all/company/type)
|
||||
• GET /api/cache/user-settings - Get user cache settings
|
||||
• POST /api/cache/user-settings - Enable/disable cache for user
|
||||
|
||||
📈 PERFORMANCE TRACKING:
|
||||
|
||||
Each cached request includes metadata:
|
||||
• cache_hit: true/false (was result from cache?)
|
||||
• cache_source: "L1" | "L2" | null (which cache tier?)
|
||||
• response_time_ms: float (total response time)
|
||||
• time_saved_ms: float (estimated time saved vs Oracle query)
|
||||
```
|
||||
|
||||
## 🔄 **FLUX DE AUTENTIFICARE**
|
||||
|
||||
```
|
||||
1. User Login (Frontend)
|
||||
↓
|
||||
2. POST /api/auth/login (Backend)
|
||||
↓
|
||||
3. Oracle Authentication via SSH Tunnel
|
||||
• pack_drepturi.verificautilizator(username, password)
|
||||
↓
|
||||
4. JWT Token Generation (Backend)
|
||||
• Access Token (30 min)
|
||||
• Refresh Token (7 days)
|
||||
• User data + companies + permissions
|
||||
↓
|
||||
5. Token Storage (Frontend - Pinia Store)
|
||||
↓
|
||||
6. Subsequent API Requests
|
||||
• Authorization: Bearer <token>
|
||||
• AuthenticationMiddleware validation
|
||||
• User data injection in request.state
|
||||
```
|
||||
|
||||
## 🚦 **MIDDLEWARE AUTHENTICATION FLOW**
|
||||
|
||||
```
|
||||
Incoming Request
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ Rate Limiting │ → 429 if exceeded (5 req/5min per IP)
|
||||
└─────┬───────────┘
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ Path Exclusion │ → Skip auth for /docs, /health, /api/auth/login
|
||||
└─────┬───────────┘
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ Token Extract │ → 401 if missing Authorization header
|
||||
└─────┬───────────┘
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ JWT Validation │ → 401 if invalid/expired/malformed
|
||||
└─────┬───────────┘
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ User Injection │ → request.state.user = CurrentUser
|
||||
└─────┬───────────┘ → request.state.is_authenticated = True
|
||||
↓ → request.state.token_data = TokenData
|
||||
┌─────────────────┐
|
||||
│ Security Headers│ → X-Content-Type-Options, X-Frame-Options
|
||||
└─────┬───────────┘ → X-XSS-Protection, X-Process-Time
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ Route Handler │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## 🗂️ **STRUCTURA DE FIȘIERE**
|
||||
|
||||
### Frontend (Vue.js)
|
||||
```
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── components/
|
||||
│ │ ├── dashboard/
|
||||
│ │ ├── layout/
|
||||
│ │ ├── reports/
|
||||
│ │ └── ui/
|
||||
│ ├── stores/ (Pinia)
|
||||
│ │ ├── auth.js
|
||||
│ │ ├── companies.js
|
||||
│ │ ├── dashboard.js
|
||||
│ │ ├── invoices.js
|
||||
│ │ └── treasury.js
|
||||
│ ├── services/
|
||||
│ │ └── api.js
|
||||
│ ├── views/
|
||||
│ │ ├── LoginView.vue
|
||||
│ │ ├── DashboardView.vue
|
||||
│ │ ├── InvoicesView.vue
|
||||
│ │ └── BankCashRegisterView.vue
|
||||
│ └── router/
|
||||
└── tests/ (Playwright E2E)
|
||||
```
|
||||
|
||||
### Backend (FastAPI)
|
||||
```
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── main.py
|
||||
│ ├── routers/
|
||||
│ │ ├── auth.py
|
||||
│ │ ├── companies.py
|
||||
│ │ ├── dashboard.py
|
||||
│ │ ├── invoices.py
|
||||
│ │ └── treasury.py
|
||||
│ ├── services/
|
||||
│ │ ├── invoice_service.py
|
||||
│ │ ├── dashboard_service.py
|
||||
│ │ └── treasury_service.py
|
||||
│ └── models/
|
||||
└── shared/
|
||||
├── auth/
|
||||
│ ├── middleware.py
|
||||
│ ├── jwt_handler.py
|
||||
│ ├── auth_service.py
|
||||
│ └── models.py
|
||||
└── database/
|
||||
└── oracle_pool.py
|
||||
```
|
||||
|
||||
## 🔧 **TEHNOLOGII UTILIZATE**
|
||||
|
||||
### Frontend Stack
|
||||
- **Vue.js 3** - Framework JavaScript reactiv
|
||||
- **PrimeVue** - UI Component Library
|
||||
- **Pinia** - State Management
|
||||
- **Vite** - Build Tool & Dev Server
|
||||
- **Axios** - HTTP Client
|
||||
- **Vue Router** - Client-side routing
|
||||
- **Chart.js** - Data visualization
|
||||
- **Playwright** - E2E Testing
|
||||
|
||||
### Backend Stack
|
||||
- **FastAPI** - Python Web Framework
|
||||
- **Uvicorn** - ASGI Server
|
||||
- **PyJWT** - JWT Token handling
|
||||
- **cx_Oracle** - Oracle Database driver
|
||||
- **Pydantic** - Data validation
|
||||
- **Python-dotenv** - Environment variables
|
||||
|
||||
### Database & Infrastructure
|
||||
- **Oracle Database** - Persistent data storage
|
||||
- **SSH Tunnel** - Secure database connection (Linux/development)
|
||||
- **Docker** - Containerization (Linux production)
|
||||
- **Nginx** - Reverse proxy & static files (Linux production)
|
||||
- **Windows Server + IIS** - Windows production deployment
|
||||
- **NSSM** - Windows service manager
|
||||
|
||||
## 🪟 **ARHITECTURA WINDOWS SERVER/IIS**
|
||||
|
||||
### Deployment pe Windows Server
|
||||
|
||||
ROA2WEB poate fi deployment pe Windows Server folosind IIS și Windows Services, fără Docker:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🌐 CLIENT │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ HTTP/HTTPS Requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🪟 IIS WEB SERVER │
|
||||
│ Port: 80/443 (HTTPS with SSL certificate) │
|
||||
│ │
|
||||
│ 📁 Static Files Serving: 🔀 URL Rewrite Module: │
|
||||
│ • Frontend (Vue.js build) • /api/* → Backend Service │
|
||||
│ • web.config configuration • /* → index.html (SPA routing) │
|
||||
│ • Compression & Caching • Application Request Routing (ARR) │
|
||||
│ │
|
||||
│ ⚙️ Application Pool: │
|
||||
│ • ROA2WEB-AppPool (.NET not required) │
|
||||
│ • Integrated pipeline mode │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ Reverse Proxy to Backend
|
||||
│ http://localhost:8000/api/*
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🔧 WINDOWS SERVICE │
|
||||
│ Service Name: ROA2WEB-Backend │
|
||||
│ Manager: NSSM (Non-Sucking Service Manager) │
|
||||
│ Port: 8000 (localhost only) │
|
||||
│ │
|
||||
│ 📊 Backend Components: │
|
||||
│ • FastAPI + Uvicorn (Python 3.11+) │
|
||||
│ • Auto-start on Windows boot │
|
||||
│ • Auto-restart on failure (5 sec delay) │
|
||||
│ • Logging to file (stdout/stderr) │
|
||||
│ │
|
||||
│ 📁 Installation Location: │
|
||||
│ • C:\inetpub\wwwroot\roa2web\backend\ │
|
||||
└─────────────────┬───────────────────────────────────────────────────────────────┘
|
||||
│ Direct Database Connection
|
||||
│ No SSH Tunnel Required
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🏛️ ORACLE DATABASE (Local/Network) │
|
||||
│ Connection: Direct TCP/IP (localhost:1521 or network) │
|
||||
│ Schema: CONTAFIN_ORACLE │
|
||||
│ │
|
||||
│ 📋 Same Tables/Views as Linux deployment │
|
||||
│ 🔧 Same Stored Procedures │
|
||||
└─────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Diferențe între Linux și Windows Deployment
|
||||
|
||||
| Aspect | Linux (Docker) | Windows (IIS) |
|
||||
|--------|----------------|---------------|
|
||||
| **Web Server** | Nginx (container) | IIS (native) |
|
||||
| **Backend Runtime** | Docker container | Windows Service (NSSM) |
|
||||
| **Database Access** | SSH Tunnel required | Direct connection |
|
||||
| **SSL/TLS** | Let's Encrypt (certbot) | IIS SSL certificates |
|
||||
| **Service Management** | Docker Compose | PowerShell + Services.msc |
|
||||
| **Deployment** | `./scripts/deploy.sh` | `Deploy-ROA2WEB.ps1` |
|
||||
| **Logs** | Docker logs | Windows Event Log + Files |
|
||||
| **Auto-start** | Docker restart policies | Windows Service auto-start |
|
||||
|
||||
### Structura Fișiere Windows Deployment
|
||||
|
||||
```
|
||||
C:\inetpub\wwwroot\roa2web\
|
||||
├── backend\ # FastAPI application
|
||||
│ ├── app\
|
||||
│ │ ├── main.py
|
||||
│ │ ├── routers\
|
||||
│ │ ├── services\
|
||||
│ │ └── models\
|
||||
│ ├── shared\ # Shared components
|
||||
│ │ ├── auth\
|
||||
│ │ ├── database\
|
||||
│ │ └── utils\
|
||||
│ ├── requirements.txt
|
||||
│ ├── .env # Production config
|
||||
│ └── logs\
|
||||
│
|
||||
├── frontend\ # Vue.js build output
|
||||
│ ├── index.html
|
||||
│ ├── assets\
|
||||
│ ├── web.config # IIS configuration
|
||||
│ └── ...
|
||||
│
|
||||
├── logs\ # Service logs
|
||||
│ ├── backend-stdout.log
|
||||
│ └── backend-stderr.log
|
||||
│
|
||||
└── backups\ # Deployment backups
|
||||
└── backup-YYYYMMDD-HHMMSS\
|
||||
```
|
||||
|
||||
### Comenzi Windows Deployment
|
||||
|
||||
```powershell
|
||||
# Instalare inițială
|
||||
.\Install-ROA2WEB.ps1
|
||||
|
||||
# Deployment actualizări
|
||||
.\Deploy-ROA2WEB.ps1 -SourcePath "C:\path\to\deploy-package"
|
||||
|
||||
# Management serviciu
|
||||
.\Start-ROA2WEB.ps1
|
||||
.\Stop-ROA2WEB.ps1
|
||||
.\Restart-ROA2WEB.ps1
|
||||
|
||||
# Verificare status
|
||||
Get-Service ROA2WEB-Backend
|
||||
Get-Website ROA2WEB
|
||||
|
||||
# Logs
|
||||
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stdout.log -Tail 50 -Wait
|
||||
```
|
||||
|
||||
Pentru detalii complete despre deployment pe Windows, consultați:
|
||||
- `/deployment/windows/docs/WINDOWS_DEPLOYMENT.md`
|
||||
- `/deployment/windows/README.md`
|
||||
|
||||
## ⚙️ **COMENZI DE DEZVOLTARE**
|
||||
|
||||
### Start SSH Tunnel
|
||||
```bash
|
||||
cd /mnt/d/PROIECTE/roa-flask/roa2web
|
||||
./ssh_tunnel.sh start
|
||||
```
|
||||
|
||||
### Backend Development
|
||||
```bash
|
||||
cd reports-app/backend/
|
||||
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
### Frontend Development
|
||||
```bash
|
||||
cd reports-app/frontend/
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### Testing
|
||||
```bash
|
||||
cd shared/
|
||||
python -m pytest -v
|
||||
```
|
||||
|
||||
## 🛡️ **SECURITATE**
|
||||
|
||||
### Middleware de Autentificare
|
||||
- **JWT Token Validation** - Verificare automată pentru toate endpoint-urile protejate
|
||||
- **Rate Limiting** - Protecție împotriva atacurilor brute force
|
||||
- **Security Headers** - X-Content-Type-Options, X-Frame-Options, X-XSS-Protection
|
||||
- **CORS Protection** - Configurare restrictivă pentru frontend-uri autorizate
|
||||
|
||||
### Baza de Date
|
||||
- **SSH Tunnel** - Conexiune criptată la Oracle
|
||||
- **Schema Dedicată** - CONTAFIN_ORACLE pentru izolare
|
||||
- **Stored Procedures** - Validare securizată de utilizatori
|
||||
|
||||
---
|
||||
|
||||
*Această schemă oferă o vedere de ansamblu asupra arhitecturii ROA2WEB și poate fi utilizată pentru documentare, onboarding și planificarea dezvoltării viitoare.*
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,400 +0,0 @@
|
||||
# ROA2WEB Docker Setup Guide
|
||||
|
||||
This guide covers how to set up and run ROA2WEB using Docker and Docker Compose for both development and production environments.
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
- Docker (20.10+)
|
||||
- Docker Compose (2.0+)
|
||||
- Git
|
||||
- 4GB+ available RAM
|
||||
- 10GB+ available disk space
|
||||
|
||||
### Windows/WSL2 Users
|
||||
- WSL2 with Ubuntu/Debian
|
||||
- Docker Desktop for Windows with WSL2 backend
|
||||
|
||||
## 🚀 Quick Start (Development)
|
||||
|
||||
### 1. Clone and Setup Environment
|
||||
|
||||
```bash
|
||||
cd
|
||||
cp .env.development .env
|
||||
```
|
||||
|
||||
### 2. Configure Database Connection
|
||||
|
||||
Edit `.env` file with your Oracle database credentials:
|
||||
|
||||
```env
|
||||
ORACLE_USER=CONTAFIN_ORACLE
|
||||
ORACLE_PASSWORD=your_password_here
|
||||
ORACLE_HOST=localhost # via SSH tunnel
|
||||
ORACLE_PORT=1521
|
||||
ORACLE_SID=ROA
|
||||
```
|
||||
|
||||
### 3. Start SSH Tunnel (if needed)
|
||||
|
||||
```bash
|
||||
./ssh_tunnel.sh start
|
||||
```
|
||||
|
||||
### 4. Build and Start Services
|
||||
|
||||
```bash
|
||||
# Build images and start services
|
||||
docker-compose up --build
|
||||
|
||||
# Or run in background
|
||||
docker-compose up -d --build
|
||||
```
|
||||
|
||||
### 5. Access the Application
|
||||
|
||||
- **Frontend**: http://localhost:8080 (via Nginx Gateway)
|
||||
- **Backend API**: http://localhost:8000 (direct access)
|
||||
- **Frontend Direct**: http://localhost:3000 (direct access)
|
||||
- **Redis**: http://localhost:6379 (direct access)
|
||||
|
||||
### 6. View Logs
|
||||
|
||||
```bash
|
||||
# All services
|
||||
docker-compose logs -f
|
||||
|
||||
# Specific service
|
||||
docker-compose logs -f roa-backend
|
||||
docker-compose logs -f roa-frontend
|
||||
docker-compose logs -f roa-gateway
|
||||
```
|
||||
|
||||
## 🏭 Production Deployment
|
||||
|
||||
### 1. Prepare Production Environment
|
||||
|
||||
```bash
|
||||
# Copy production template
|
||||
cp .env.production .env.production.local
|
||||
|
||||
# Edit with your production values
|
||||
nano .env.production.local
|
||||
```
|
||||
|
||||
### 2. Create Production Secrets
|
||||
|
||||
```bash
|
||||
# Create secrets directory
|
||||
mkdir -p secrets/
|
||||
|
||||
# Add your production secrets
|
||||
echo "your_oracle_password" > secrets/oracle_password.txt
|
||||
echo "your_jwt_secret_key" > secrets/jwt_secret_key.txt
|
||||
echo "your_redis_password" > secrets/redis_password.txt
|
||||
|
||||
# Secure the secrets
|
||||
chmod 600 secrets/*.txt
|
||||
```
|
||||
|
||||
### 3. Configure SSL Domain
|
||||
|
||||
Update `.env.production.local`:
|
||||
|
||||
```env
|
||||
DOMAIN=your-domain.com
|
||||
SSL_EMAIL=admin@your-domain.com
|
||||
```
|
||||
|
||||
### 4. Deploy to Production
|
||||
|
||||
```bash
|
||||
# Using deployment script (recommended)
|
||||
./scripts/deploy.sh
|
||||
|
||||
# Or manually
|
||||
docker-compose -f docker-compose.yml -f docker-compose.production.yml up -d --build
|
||||
```
|
||||
|
||||
### 5. Verify Deployment
|
||||
|
||||
```bash
|
||||
# Check services health
|
||||
./scripts/health-check.sh
|
||||
|
||||
# Check individual services
|
||||
curl http://localhost/health
|
||||
curl http://localhost/api/health
|
||||
```
|
||||
|
||||
## 🛠️ Development Workflow
|
||||
|
||||
### Hot Reload Development
|
||||
|
||||
The development setup includes hot reload for both frontend and backend:
|
||||
|
||||
```bash
|
||||
# Start with override (development config)
|
||||
docker-compose up
|
||||
|
||||
# Backend code changes in reports-app/backend/app/ are reflected immediately
|
||||
# Frontend code changes in reports-app/frontend/src/ trigger rebuild
|
||||
```
|
||||
|
||||
### Database Changes
|
||||
|
||||
```bash
|
||||
# Restart backend after database schema changes
|
||||
docker-compose restart roa-backend
|
||||
|
||||
# View backend logs
|
||||
docker-compose logs -f roa-backend
|
||||
```
|
||||
|
||||
### Frontend Development
|
||||
|
||||
```bash
|
||||
# Rebuild frontend after package changes
|
||||
docker-compose build roa-frontend
|
||||
docker-compose up -d roa-frontend
|
||||
|
||||
# Access frontend directly for debugging
|
||||
# http://localhost:3000
|
||||
```
|
||||
|
||||
## 📊 Monitoring and Maintenance
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Comprehensive health check
|
||||
./scripts/health-check.sh full
|
||||
|
||||
# Quick service check
|
||||
./scripts/health-check.sh quick
|
||||
|
||||
# Continuous monitoring
|
||||
./scripts/health-check.sh watch
|
||||
```
|
||||
|
||||
### Backup and Restore
|
||||
|
||||
```bash
|
||||
# Full backup
|
||||
./scripts/backup.sh full
|
||||
|
||||
# Database only
|
||||
./scripts/backup.sh database
|
||||
|
||||
# List backups
|
||||
./scripts/backup.sh list
|
||||
|
||||
# Restore from backup
|
||||
./scripts/backup.sh restore backup_20240131_143022
|
||||
```
|
||||
|
||||
### Log Management
|
||||
|
||||
```bash
|
||||
# View real-time logs
|
||||
docker-compose logs -f
|
||||
|
||||
# View logs with timestamps
|
||||
docker-compose logs -t
|
||||
|
||||
# Export logs
|
||||
docker-compose logs > roa2web_logs_$(date +%Y%m%d).log
|
||||
```
|
||||
|
||||
## 🔧 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Port Already in Use
|
||||
|
||||
```bash
|
||||
# Check what's using the port
|
||||
sudo netstat -tlnp | grep :8080
|
||||
|
||||
# Stop the conflicting service or change ports in docker-compose.override.yml
|
||||
```
|
||||
|
||||
#### 2. Database Connection Failed
|
||||
|
||||
```bash
|
||||
# Check SSH tunnel status
|
||||
./ssh_tunnel.sh status
|
||||
|
||||
# Restart SSH tunnel
|
||||
./ssh_tunnel.sh restart
|
||||
|
||||
# Test database connection
|
||||
docker-compose exec roa-backend python -c "from shared.database.oracle_pool import test_connection; test_connection()"
|
||||
```
|
||||
|
||||
#### 3. Frontend Build Errors
|
||||
|
||||
```bash
|
||||
# Clear node_modules and rebuild
|
||||
docker-compose build --no-cache roa-frontend
|
||||
|
||||
# Check frontend logs
|
||||
docker-compose logs roa-frontend
|
||||
```
|
||||
|
||||
#### 4. SSL Certificate Issues (Production)
|
||||
|
||||
```bash
|
||||
# Generate test certificates
|
||||
docker-compose exec roa-gateway /usr/local/bin/ssl-renew.sh
|
||||
|
||||
# Check certificate status
|
||||
docker-compose exec roa-gateway openssl x509 -in /etc/letsencrypt/live/your-domain.com/cert.pem -text -noout
|
||||
```
|
||||
|
||||
### Service Recovery
|
||||
|
||||
#### Quick Recovery
|
||||
|
||||
```bash
|
||||
# Restart all services
|
||||
docker-compose restart
|
||||
|
||||
# Rollback to previous version
|
||||
./scripts/rollback.sh quick
|
||||
```
|
||||
|
||||
#### Full Recovery
|
||||
|
||||
```bash
|
||||
# Stop everything
|
||||
docker-compose down
|
||||
|
||||
# Clean up
|
||||
docker system prune -f
|
||||
|
||||
# Restart fresh
|
||||
docker-compose up -d --build
|
||||
```
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
#### Development
|
||||
|
||||
```bash
|
||||
# Allocate more memory to Docker
|
||||
# Docker Desktop: Settings > Resources > Memory (recommend 4GB+)
|
||||
|
||||
# Disable unnecessary services in development
|
||||
# Comment out services in docker-compose.override.yml
|
||||
```
|
||||
|
||||
#### Production
|
||||
|
||||
```bash
|
||||
# Monitor resource usage
|
||||
docker stats
|
||||
|
||||
# Scale services
|
||||
docker-compose -f docker-compose.yml -f docker-compose.production.yml up -d --scale roa-backend=2
|
||||
|
||||
# Optimize images
|
||||
docker image prune -f
|
||||
docker volume prune -f
|
||||
```
|
||||
|
||||
## 🔒 Security
|
||||
|
||||
### Development Security
|
||||
|
||||
- Never commit actual credentials to version control
|
||||
- Use `.env` files that are gitignored
|
||||
- SSH tunnel provides secure database access
|
||||
|
||||
### Production Security
|
||||
|
||||
- Use Docker secrets for sensitive data
|
||||
- Enable SSL/TLS with valid certificates
|
||||
- Regular security updates
|
||||
- Monitor logs for suspicious activity
|
||||
|
||||
```bash
|
||||
# Update base images
|
||||
docker-compose pull
|
||||
docker-compose up -d --build
|
||||
|
||||
# Security scan
|
||||
docker scout cves backend:latest
|
||||
```
|
||||
|
||||
## 📚 Advanced Configuration
|
||||
|
||||
### Custom Nginx Configuration
|
||||
|
||||
Edit `nginx/conf/sites-enabled/roa2web.conf` for custom routing:
|
||||
|
||||
```nginx
|
||||
# Add custom location
|
||||
location /custom-api/ {
|
||||
proxy_pass http://custom-service:3000/;
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
```
|
||||
|
||||
### Environment-Specific Overrides
|
||||
|
||||
Create custom compose files:
|
||||
|
||||
```yaml
|
||||
# docker-compose.staging.yml
|
||||
version: '3.8'
|
||||
services:
|
||||
roa-backend:
|
||||
environment:
|
||||
- DEBUG=false
|
||||
- LOG_LEVEL=INFO
|
||||
```
|
||||
|
||||
### Adding New Services
|
||||
|
||||
```yaml
|
||||
# Add to docker-compose.yml
|
||||
services:
|
||||
new-service:
|
||||
build: ./new-service
|
||||
networks:
|
||||
- roa-network
|
||||
depends_on:
|
||||
- roa-backend
|
||||
```
|
||||
|
||||
## 📞 Support
|
||||
|
||||
### Getting Help
|
||||
|
||||
1. Check logs: `docker-compose logs`
|
||||
2. Run health check: `./scripts/health-check.sh`
|
||||
3. Review this documentation
|
||||
4. Check GitHub issues
|
||||
5. Contact the development team
|
||||
|
||||
### Useful Commands Reference
|
||||
|
||||
```bash
|
||||
# Quick commands
|
||||
docker-compose up -d # Start services in background
|
||||
docker-compose down # Stop and remove containers
|
||||
docker-compose ps # Show running services
|
||||
docker-compose exec roa-backend sh # Access backend container
|
||||
|
||||
# Maintenance
|
||||
docker system df # Show Docker disk usage
|
||||
docker system prune -f # Clean up unused resources
|
||||
docker-compose pull # Update base images
|
||||
docker-compose build --no-cache # Rebuild without cache
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Last updated: $(date +%Y-%m-%d)*
|
||||
*ROA2WEB Docker Setup Guide v1.0*
|
||||
@@ -1,234 +0,0 @@
|
||||
# ROA2WEB Microservices Guide
|
||||
|
||||
🚀 **Ghid pentru Adăugarea de Module Noi în Ecosistemul ROA2WEB**
|
||||
|
||||
## 📋 Conceptul Microserviciilor
|
||||
|
||||
ROA2WEB folosește o arhitectură microservicii care permite adăugarea ușoară de module noi fără a afecta funcționalitatea existentă.
|
||||
|
||||
### 🏗️ Structura unui Microserviciu
|
||||
|
||||
```
|
||||
new-app/
|
||||
├── backend/ # FastAPI Backend
|
||||
│ ├── app/
|
||||
│ │ ├── main.py # Entry point
|
||||
│ │ ├── models/ # Pydantic models
|
||||
│ │ ├── routers/ # API endpoints
|
||||
│ │ ├── services/ # Business logic
|
||||
│ │ └── schemas/ # Response schemas
|
||||
│ ├── requirements.txt
|
||||
│ ├── Dockerfile
|
||||
│ └── .env.example
|
||||
├── frontend/ # Vue.js Frontend (opțional)
|
||||
│ ├── src/
|
||||
│ │ ├── main.js
|
||||
│ │ ├── App.vue
|
||||
│ │ ├── router/
|
||||
│ │ ├── stores/
|
||||
│ │ ├── views/
|
||||
│ │ └── components/
|
||||
│ ├── package.json
|
||||
│ ├── vite.config.js
|
||||
│ └── Dockerfile
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## 🔧 Shared Components
|
||||
|
||||
Toate microserviciile folosesc componentele comune din directorul `shared/`:
|
||||
|
||||
### Database Pool
|
||||
```python
|
||||
from shared.database.oracle_pool import oracle_pool
|
||||
|
||||
async with oracle_pool.get_connection() as conn:
|
||||
# Database operations
|
||||
```
|
||||
|
||||
### Authentication
|
||||
```python
|
||||
from shared.auth.jwt_handler import jwt_handler
|
||||
from shared.auth.middleware import require_auth
|
||||
|
||||
@require_auth
|
||||
async def protected_endpoint():
|
||||
# Protected logic
|
||||
```
|
||||
|
||||
## 🚀 Pași pentru Adăugare Microserviciu Nou
|
||||
|
||||
### 1. Creare Structură
|
||||
|
||||
```bash
|
||||
mkdir -p new-app/{backend/app/{models,routers,services,schemas},frontend/src/{router,stores,views,components}}
|
||||
```
|
||||
|
||||
### 2. Backend Setup
|
||||
|
||||
**`new-app/backend/app/main.py`**:
|
||||
```python
|
||||
from fastapi import FastAPI
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add shared path
|
||||
sys.path.append(os.path.join(os.path.dirname(__file__), '../../../shared'))
|
||||
|
||||
from database.oracle_pool import oracle_pool
|
||||
from auth.jwt_handler import jwt_handler
|
||||
|
||||
app = FastAPI(title="New App API")
|
||||
|
||||
@app.on_event("startup")
|
||||
async def startup():
|
||||
await oracle_pool.initialize()
|
||||
|
||||
@app.on_event("shutdown")
|
||||
async def shutdown():
|
||||
await oracle_pool.close_pool()
|
||||
```
|
||||
|
||||
### 3. Frontend Setup (Opțional)
|
||||
|
||||
Dacă microserviciul necesită UI:
|
||||
|
||||
**`new-app/frontend/package.json`**:
|
||||
```json
|
||||
{
|
||||
"name": "new-app-frontend",
|
||||
"version": "1.0.0",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vite build"
|
||||
},
|
||||
"dependencies": {
|
||||
"vue": "^3.3.0",
|
||||
"primevue": "^3.0.0",
|
||||
"pinia": "^2.0.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Docker Configuration
|
||||
|
||||
**`new-app/backend/Dockerfile`**:
|
||||
```dockerfile
|
||||
FROM python:3.9-slim
|
||||
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
### 5. Nginx Routing
|
||||
|
||||
Adaugă în `nginx/nginx.conf`:
|
||||
|
||||
```nginx
|
||||
location /new-app/ {
|
||||
proxy_pass http://new-app-backend:8000/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Docker Compose Integration
|
||||
|
||||
Adaugă în `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
new-app-backend:
|
||||
build: ./new-app/backend
|
||||
networks:
|
||||
- roa-network
|
||||
environment:
|
||||
- ORACLE_USER=${ORACLE_USER}
|
||||
- ORACLE_PASSWORD=${ORACLE_PASSWORD}
|
||||
- ORACLE_DSN=${ORACLE_DSN}
|
||||
```
|
||||
|
||||
## 📊 Exemple de Microservicii
|
||||
|
||||
### 1. Invoicing App
|
||||
- Gestionare facturi
|
||||
- Generare PDF
|
||||
- Email notifications
|
||||
|
||||
### 2. Inventory App
|
||||
- Gestiune stocuri
|
||||
- Mișcări de marfă
|
||||
- Rapoarte inventar
|
||||
|
||||
### 3. CRM App
|
||||
- Gestionare clienți
|
||||
- Istoric interacțiuni
|
||||
- Pipeline vânzări
|
||||
|
||||
## 🔐 Securitate
|
||||
|
||||
### Autentificare
|
||||
Toate microserviciile folosesc JWT tokens din `shared/auth/`.
|
||||
|
||||
### Autorizare
|
||||
Implementează middleware pentru verificarea permisiunilor per modul.
|
||||
|
||||
### Database Access
|
||||
Folosește doar `shared/database/oracle_pool.py` pentru acces la baza de date.
|
||||
|
||||
## 📋 Best Practices
|
||||
|
||||
### 1. Naming Convention
|
||||
- **Directoare**: `kebab-case` (ex: `invoicing-app`)
|
||||
- **API Endpoints**: `/api/v1/resource`
|
||||
- **Database**: Schema separată per modul
|
||||
|
||||
### 2. Error Handling
|
||||
```python
|
||||
from shared.utils.exceptions import ROAException
|
||||
|
||||
@app.exception_handler(ROAException)
|
||||
async def roa_exception_handler(request, exc):
|
||||
return {"error": str(exc)}
|
||||
```
|
||||
|
||||
### 3. Logging
|
||||
```python
|
||||
import logging
|
||||
logger = logging.getLogger(f"roa.{__name__}")
|
||||
```
|
||||
|
||||
### 4. Testing
|
||||
```bash
|
||||
# Unit tests
|
||||
pytest new-app/backend/tests/
|
||||
|
||||
# Integration tests
|
||||
pytest tests/integration/test_new_app.py
|
||||
```
|
||||
|
||||
## 🔄 Deployment
|
||||
|
||||
### Development
|
||||
```bash
|
||||
docker-compose up new-app-backend new-app-frontend
|
||||
```
|
||||
|
||||
### Production
|
||||
Folosește orchestratoare precum Kubernetes pentru scalare automată.
|
||||
|
||||
## 📞 Support
|
||||
|
||||
Pentru întrebări despre dezvoltarea de microservicii:
|
||||
|
||||
1. Consultă documentația shared components
|
||||
2. Urmărește pattern-urile din `reports-app/`
|
||||
3. Testează integrarea cu componentele comune
|
||||
|
||||
---
|
||||
|
||||
*Arhitectura microservicii permite creșterea organică a platformei ROA2WEB* 🚀
|
||||
564
docs/MONOLITH_ARCHITECTURE.md
Normal file
564
docs/MONOLITH_ARCHITECTURE.md
Normal file
@@ -0,0 +1,564 @@
|
||||
# ROA2WEB Ultrathin Monolith Architecture
|
||||
|
||||
**Version:** 1.0.0
|
||||
**Last Updated:** 2025-12-29
|
||||
**Status:** ✅ Active
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
ROA2WEB uses an **ultrathin monolith** architecture - a single unified backend and frontend application with modular organization. This architecture provides the simplicity of a monolith with the organizational benefits of microservices.
|
||||
|
||||
### Key Characteristics
|
||||
|
||||
- **Single Backend Process**: One FastAPI application serving all modules
|
||||
- **Single Frontend Build**: One Vue.js SPA with lazy-loaded modules
|
||||
- **Modular Organization**: Business logic separated into distinct modules
|
||||
- **Shared Resources**: Database pools, auth, cache shared across modules
|
||||
- **Independent Modules**: Each module can be developed and tested independently
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🌐 CLIENT BROWSER │
|
||||
└────────────────────────────────┬────────────────────────────────────────────┘
|
||||
│ HTTP/HTTPS Requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 📦 UNIFIED VUE.JS FRONTEND │
|
||||
│ Single-page application (SPA) served from /dist │
|
||||
│ Port: 3000 (dev) / 80,443 (production) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Frontend Modules (Lazy Loaded) │ │
|
||||
│ │ │ │
|
||||
│ │ • src/modules/reports/ - Reports module UI │ │
|
||||
│ │ • src/modules/data-entry/ - Data entry module UI │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Shared Frontend Components │ │
|
||||
│ │ │ │
|
||||
│ │ • src/shared/components/ - Reusable Vue components │ │
|
||||
│ │ • src/shared/stores/ - Pinia stores │ │
|
||||
│ │ • src/assets/css/ - Global CSS system │ │
|
||||
│ │ • shared/frontend/ - Login, auth components │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────┬────────────────────────────────────────────┘
|
||||
│ API Calls (Axios)
|
||||
│ Authorization: Bearer <JWT>
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🚀 UNIFIED FASTAPI BACKEND │
|
||||
│ Single Python process running all modules │
|
||||
│ Port: 8000 (dev) / 8001 (production) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 🛡️ Global Middleware Layer │ │
|
||||
│ │ │ │
|
||||
│ │ • CORSMiddleware - CORS handling │ │
|
||||
│ │ • AuthenticationMiddleware - JWT validation & user injection │ │
|
||||
│ │ • Rate Limiting - 5 req/5 min per IP │ │
|
||||
│ │ • Security Headers - XSS, CSP, Frame protection │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 📊 Backend Modules (backend/modules/) │ │
|
||||
│ │ │ │
|
||||
│ │ reports/ │ │
|
||||
│ │ ├── routers/ - API endpoints │ │
|
||||
│ │ ├── services/ - Business logic with caching │ │
|
||||
│ │ ├── models/ - Pydantic models │ │
|
||||
│ │ └── cache/ - L1+L2 cache implementation │ │
|
||||
│ │ │ │
|
||||
│ │ data_entry/ │ │
|
||||
│ │ ├── routers/ - API endpoints │ │
|
||||
│ │ ├── services/ - Business logic │ │
|
||||
│ │ ├── models/ - SQLModel ORM models │ │
|
||||
│ │ └── db/ - SQLite database │ │
|
||||
│ │ │ │
|
||||
│ │ telegram/ │ │
|
||||
│ │ ├── bot/ - Telegram bot handlers │ │
|
||||
│ │ ├── api/ - Internal API for bot │ │
|
||||
│ │ └── db/ - Bot session management │ │
|
||||
│ │ │ │
|
||||
│ │ data/ │ │
|
||||
│ │ └── telegram_bot.db - Telegram bot SQLite database │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ 🔧 Shared Backend Components (shared/) │ │
|
||||
│ │ │ │
|
||||
│ │ database/ │ │
|
||||
│ │ └── oracle_pool.py - Singleton Oracle connection pool │ │
|
||||
│ │ │ │
|
||||
│ │ auth/ │ │
|
||||
│ │ ├── middleware.py - JWT authentication middleware │ │
|
||||
│ │ ├── jwt_handler.py - Token creation/validation │ │
|
||||
│ │ └── models.py - Auth data models │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────┬────────────────────────────────────────────┘
|
||||
│ Database Queries
|
||||
│ (Oracle + SQLite)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ 🗄️ DATABASE LAYER │
|
||||
│ │
|
||||
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ Oracle Database │ │ SQLite Databases │ │
|
||||
│ │ (via SSH Tunnel) │ │ (Local Files) │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ • CONTAFIN_ORACLE │ │ • data_entry.db │ │
|
||||
│ │ • Financial schemas │ │ • telegram_bot.db │ │
|
||||
│ │ • User management │ │ • cache.db │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ Port: 1521 (remote) │ │ Path: backend/modules/ │ │
|
||||
│ │ 1526 (tunnel) │ │ /data/ │ │
|
||||
│ │ │ │ │ │
|
||||
│ └─────────────────────────┘ └─────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Directory Structure
|
||||
|
||||
### Backend Structure
|
||||
|
||||
```
|
||||
backend/
|
||||
├── main.py # FastAPI app entry point
|
||||
├── config.py # Centralized configuration
|
||||
├── .env # Environment variables
|
||||
├── .env.dev # Development environment
|
||||
├── .env.test # Test environment
|
||||
├── .env.prod # Production environment
|
||||
│
|
||||
└── modules/ # Business logic modules
|
||||
├── __init__.py
|
||||
│
|
||||
├── reports/ # Reports module (Oracle read-only)
|
||||
│ ├── __init__.py
|
||||
│ ├── routers/ # API endpoints
|
||||
│ │ ├── dashboard.py
|
||||
│ │ ├── invoices.py
|
||||
│ │ └── treasury.py
|
||||
│ ├── services/ # Business logic
|
||||
│ │ ├── dashboard_service.py
|
||||
│ │ └── invoice_service.py
|
||||
│ ├── models/ # Pydantic models
|
||||
│ ├── schemas/ # Response schemas
|
||||
│ └── cache/ # Cache implementation
|
||||
│ ├── decorators.py # @cached decorator
|
||||
│ └── manager.py # L1+L2 cache manager
|
||||
│
|
||||
├── data_entry/ # Data entry module (SQLite + workflow)
|
||||
│ ├── __init__.py
|
||||
│ ├── routers/ # API endpoints
|
||||
│ │ ├── receipts.py
|
||||
│ │ └── entries.py
|
||||
│ ├── services/ # Business logic
|
||||
│ ├── models/ # SQLModel ORM models
|
||||
│ │ ├── receipt.py
|
||||
│ │ ├── attachment.py
|
||||
│ │ └── accounting_entry.py
|
||||
│ ├── db/ # Database
|
||||
│ │ ├── session.py # SQLite session
|
||||
│ │ └── migrations/ # Alembic migrations
|
||||
│ └── utils/ # Helper functions
|
||||
│
|
||||
├── telegram/ # Telegram bot module
|
||||
│ ├── __init__.py
|
||||
│ ├── bot/ # Bot implementation
|
||||
│ │ ├── handlers.py # Command handlers
|
||||
│ │ ├── helpers.py # Helper functions
|
||||
│ │ └── formatters.py # Message formatters
|
||||
│ ├── api/ # Internal API
|
||||
│ │ └── auth.py # Auth code management
|
||||
│ ├── db/ # Bot database
|
||||
│ │ └── session.py # Session management
|
||||
│ └── bot_main.py # Bot entry point
|
||||
│
|
||||
└── data/ # Shared data
|
||||
└── telegram_bot.db # Telegram bot database
|
||||
```
|
||||
|
||||
### Frontend Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.js # App entry point
|
||||
├── App.vue # Root component
|
||||
│
|
||||
├── modules/ # Feature modules
|
||||
│ ├── reports/ # Reports module
|
||||
│ │ ├── views/ # Page components
|
||||
│ │ │ ├── DashboardView.vue
|
||||
│ │ │ ├── InvoicesView.vue
|
||||
│ │ │ └── TreasuryView.vue
|
||||
│ │ ├── components/ # Module-specific components
|
||||
│ │ ├── stores/ # Module-specific Pinia stores
|
||||
│ │ │ ├── dashboard.js
|
||||
│ │ │ └── invoices.js
|
||||
│ │ ├── services/ # API client
|
||||
│ │ │ └── api.js
|
||||
│ │ └── utils/ # Helpers
|
||||
│ │
|
||||
│ └── data-entry/ # Data entry module
|
||||
│ ├── views/ # Page components
|
||||
│ │ ├── ReceiptsView.vue
|
||||
│ │ └── EntriesView.vue
|
||||
│ ├── components/ # Module-specific components
|
||||
│ ├── stores/ # Module-specific Pinia stores
|
||||
│ │ └── receipts.js
|
||||
│ └── services/ # API client
|
||||
│ └── api.js
|
||||
│
|
||||
├── shared/ # Shared components
|
||||
│ ├── components/ # Reusable Vue components
|
||||
│ │ ├── ErrorBoundary.vue
|
||||
│ │ ├── CompanySelector.vue
|
||||
│ │ └── PeriodSelector.vue
|
||||
│ └── stores/ # Shared Pinia stores
|
||||
│ ├── auth.js
|
||||
│ ├── companies.js
|
||||
│ └── accountingPeriod.js
|
||||
│
|
||||
├── router/ # Vue Router
|
||||
│ ├── index.js # Main router
|
||||
│ ├── reports.js # Reports routes
|
||||
│ └── data-entry.js # Data entry routes
|
||||
│
|
||||
├── assets/ # Global assets
|
||||
│ ├── css/ # Global CSS system
|
||||
│ │ ├── core/ # Design tokens
|
||||
│ │ ├── components/ # Component styles
|
||||
│ │ ├── patterns/ # UI patterns
|
||||
│ │ ├── layout/ # Layout styles
|
||||
│ │ ├── utilities/ # Utility classes
|
||||
│ │ └── vendor/ # PrimeVue overrides
|
||||
│ └── images/ # Images
|
||||
│
|
||||
└── views/ # Global views
|
||||
└── LoginView.vue # Login page
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Request Flow
|
||||
|
||||
### 1. User Authentication Flow
|
||||
|
||||
```
|
||||
User Login Request
|
||||
↓
|
||||
Frontend (LoginView.vue)
|
||||
↓ POST /api/auth/login
|
||||
Backend (main.py)
|
||||
↓
|
||||
AuthenticationMiddleware (excluded for /login)
|
||||
↓
|
||||
Auth Router (/api/auth/login)
|
||||
↓
|
||||
Oracle Database (pack_drepturi.verificautilizator)
|
||||
↓
|
||||
JWT Token Generation
|
||||
↓
|
||||
Response with access_token + refresh_token
|
||||
↓
|
||||
Frontend stores token in localStorage
|
||||
↓
|
||||
Subsequent requests include: Authorization: Bearer <token>
|
||||
```
|
||||
|
||||
### 2. Reports Data Flow (with Cache)
|
||||
|
||||
```
|
||||
User Request (Dashboard data)
|
||||
↓
|
||||
Frontend (DashboardView.vue)
|
||||
↓ GET /api/reports/dashboard
|
||||
Backend Middleware
|
||||
├─ CORS check ✓
|
||||
├─ JWT validation ✓
|
||||
└─ User injection into request.state
|
||||
↓
|
||||
Reports Router (/api/reports/dashboard)
|
||||
↓
|
||||
DashboardService.get_dashboard_data()
|
||||
↓
|
||||
@cached decorator checks cache
|
||||
├─ L1 Cache (Memory) HIT? → Return cached data
|
||||
├─ L2 Cache (SQLite) HIT? → Store in L1, return data
|
||||
└─ MISS → Query Oracle
|
||||
↓
|
||||
Oracle Database (via connection pool)
|
||||
↓
|
||||
Process and cache result (L1 + L2)
|
||||
↓
|
||||
Return data
|
||||
↓
|
||||
Response with cache metadata headers
|
||||
↓
|
||||
Frontend updates Pinia store
|
||||
↓
|
||||
Vue component reactively updates UI
|
||||
```
|
||||
|
||||
### 3. Data Entry Flow (with Workflow)
|
||||
|
||||
```
|
||||
User Creates Receipt
|
||||
↓
|
||||
Frontend (ReceiptsView.vue)
|
||||
↓ POST /api/data-entry/receipts
|
||||
Backend Middleware (JWT validation)
|
||||
↓
|
||||
Data Entry Router
|
||||
↓
|
||||
ReceiptService.create_receipt()
|
||||
↓
|
||||
SQLModel ORM
|
||||
↓
|
||||
SQLite Database (data_entry.db)
|
||||
├─ Insert Receipt (status: DRAFT)
|
||||
├─ Insert Attachments
|
||||
└─ Generate AccountingEntries
|
||||
↓
|
||||
Response with receipt ID
|
||||
↓
|
||||
Frontend updates Pinia store
|
||||
↓
|
||||
UI shows new receipt in DRAFT state
|
||||
↓
|
||||
User submits for approval
|
||||
↓ POST /api/data-entry/receipts/{id}/submit
|
||||
Backend updates status: DRAFT → PENDING
|
||||
↓
|
||||
Workflow continues (PENDING → APPROVED/REJECTED)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Module Independence
|
||||
|
||||
### Benefits of Modular Monolith
|
||||
|
||||
1. **Development**:
|
||||
- Each module can be developed independently
|
||||
- Clear boundaries between features
|
||||
- Easy to understand and navigate
|
||||
|
||||
2. **Testing**:
|
||||
- Modules can be tested in isolation
|
||||
- Shared components tested once
|
||||
- Integration tests verify module interactions
|
||||
|
||||
3. **Deployment**:
|
||||
- Single deployment process
|
||||
- No service coordination complexity
|
||||
- Atomic updates (all or nothing)
|
||||
|
||||
4. **Performance**:
|
||||
- No network latency between modules
|
||||
- Shared connection pools
|
||||
- Efficient resource utilization
|
||||
|
||||
### Module Communication
|
||||
|
||||
**Within Backend**:
|
||||
- Modules can directly import from each other
|
||||
- Use shared utilities from `shared/`
|
||||
- Share database connections and auth
|
||||
|
||||
**Within Frontend**:
|
||||
- Modules use shared Pinia stores
|
||||
- Shared components in `src/shared/`
|
||||
- Routing handles module navigation
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Configuration Management
|
||||
|
||||
### Environment-Based Configuration
|
||||
|
||||
The backend supports multiple environment configurations:
|
||||
|
||||
```bash
|
||||
# Development
|
||||
backend/.env.dev # Local development settings
|
||||
PORT=8000
|
||||
DEBUG=True
|
||||
ORACLE_HOST=localhost
|
||||
ORACLE_PORT=1526
|
||||
|
||||
# Testing
|
||||
backend/.env.test # Testing environment
|
||||
PORT=8001
|
||||
DEBUG=False
|
||||
ORACLE_HOST=10.0.20.121
|
||||
|
||||
# Production
|
||||
backend/.env.prod # Production settings
|
||||
PORT=8001
|
||||
DEBUG=False
|
||||
ORACLE_HOST=10.0.20.36
|
||||
```
|
||||
|
||||
### Module Toggle Configuration
|
||||
|
||||
Modules can be enabled/disabled via environment variables:
|
||||
|
||||
```bash
|
||||
MODULE_REPORTS_ENABLED=true
|
||||
MODULE_DATA_ENTRY_ENABLED=true
|
||||
MODULE_TELEGRAM_ENABLED=true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Shared Resources
|
||||
|
||||
### 1. Database Connection Pool
|
||||
|
||||
**Oracle Pool** (`shared/database/oracle_pool.py`):
|
||||
- Singleton pattern
|
||||
- Configured pool size (min=2, max=10)
|
||||
- Automatic reconnection
|
||||
- Connection health checks
|
||||
- Used by Reports module
|
||||
|
||||
### 2. Authentication & Authorization
|
||||
|
||||
**JWT Authentication** (`shared/auth/`):
|
||||
- Centralized token management
|
||||
- Middleware auto-injection of user
|
||||
- Rate limiting (5 req/5 min)
|
||||
- Used by all modules
|
||||
|
||||
### 3. Cache System (Reports Module)
|
||||
|
||||
**Two-Tier Cache**:
|
||||
- L1: In-memory (LRU, 1000 entries)
|
||||
- L2: SQLite persistent cache
|
||||
- Automatic TTL management
|
||||
- Performance tracking
|
||||
|
||||
### 4. Frontend Shared Components
|
||||
|
||||
**Shared UI** (`src/shared/` and `shared/frontend/`):
|
||||
- Authentication components
|
||||
- Company/Period selectors
|
||||
- Error boundaries
|
||||
- Global CSS system
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Migration from Microservices
|
||||
|
||||
### What Changed
|
||||
|
||||
**Before (Microservices)**:
|
||||
```
|
||||
reports-app/backend/ → Port 8001
|
||||
data-entry-app/backend/ → Port 8003
|
||||
telegram-bot/ → Port 8002
|
||||
```
|
||||
|
||||
**After (Monolith)**:
|
||||
```
|
||||
backend/ → Single port 8000/8001
|
||||
└── modules/
|
||||
├── reports/
|
||||
├── data_entry/
|
||||
└── telegram/
|
||||
```
|
||||
|
||||
### Migration Benefits
|
||||
|
||||
1. ✅ Simpler deployment (1 service instead of 3)
|
||||
2. ✅ Faster startup (no multi-service coordination)
|
||||
3. ✅ Easier debugging (single process)
|
||||
4. ✅ Shared connection pools (better resource utilization)
|
||||
5. ✅ No inter-service network latency
|
||||
6. ✅ Atomic deployments (all modules update together)
|
||||
|
||||
### What Stayed the Same
|
||||
|
||||
- ✅ Module boundaries and organization
|
||||
- ✅ Business logic and models
|
||||
- ✅ Frontend structure
|
||||
- ✅ Database schemas
|
||||
- ✅ API contracts
|
||||
- ✅ Authentication flow
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Considerations
|
||||
|
||||
### Middleware Stack
|
||||
|
||||
All requests pass through:
|
||||
1. CORS validation
|
||||
2. JWT authentication
|
||||
3. Rate limiting
|
||||
4. Security headers injection
|
||||
|
||||
### Module Isolation
|
||||
|
||||
While modules share the same process:
|
||||
- Each has its own routers
|
||||
- Business logic is separated
|
||||
- Database access is controlled
|
||||
- Permissions are enforced at API level
|
||||
|
||||
---
|
||||
|
||||
## 📈 Scalability
|
||||
|
||||
### Horizontal Scaling
|
||||
|
||||
The monolith can be scaled horizontally:
|
||||
- Multiple instances behind load balancer
|
||||
- Stateless design (JWT, no sessions)
|
||||
- Shared database (Oracle)
|
||||
- Redis for distributed cache (future)
|
||||
|
||||
### Vertical Scaling
|
||||
|
||||
Optimize single instance:
|
||||
- Increase worker processes
|
||||
- Connection pool tuning
|
||||
- Cache optimization
|
||||
- Query optimization
|
||||
|
||||
### Future: Microservices Migration
|
||||
|
||||
If needed, modules can be extracted:
|
||||
- Each module already has clear boundaries
|
||||
- Services are independent
|
||||
- API contracts are defined
|
||||
- Database can be split (if needed)
|
||||
|
||||
---
|
||||
|
||||
## 📚 Related Documentation
|
||||
|
||||
- **Setup & Commands**: `README.md`
|
||||
- **CSS Architecture**: `docs/ONBOARDING_CSS.md`
|
||||
- **Data Entry Module**: `docs/data-entry/ARCHITECTURE.md`
|
||||
- **Telegram Bot**: `docs/telegram/README.md`
|
||||
- **Deployment**: `deployment/windows/README.md` (Windows) or `DOKPLOY_DEPLOYMENT.md` (Linux)
|
||||
|
||||
---
|
||||
|
||||
**For questions about this architecture, consult the development team or open an issue in the repository.**
|
||||
@@ -1,717 +0,0 @@
|
||||
# OCR Implementation Plan - Data Entry App
|
||||
|
||||
> **Context Handover Document**
|
||||
> Created: 2025-12-11
|
||||
> Branch: `feature/data-entry-receipts`
|
||||
> Status: Ready for implementation
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Implementare OCR 100% local (fără costuri externe) pentru extragerea automată a datelor din bonuri fiscale/chitanțe românești. Soluția folosește PaddleOCR + regex extraction cu full-auto completion a formularului.
|
||||
|
||||
**Cerințe utilizator:**
|
||||
- Open-source local, fără costuri externe
|
||||
- Full-auto: completează formularul automat
|
||||
- Input: doar imagini (JPG/PNG/PDF)
|
||||
- On-premise processing
|
||||
|
||||
---
|
||||
|
||||
## Stack Tehnic Recomandat
|
||||
|
||||
| Component | Soluție | Justificare |
|
||||
|-----------|---------|-------------|
|
||||
| **OCR Engine** | PaddleOCR (primar) | 85-92% acuratețe, pip install simplu, CPU-friendly |
|
||||
| **Fallback OCR** | Tesseract + ron | Suport excelent diacritice românești |
|
||||
| **Extracție** | Regex/rules-based | Zero dependențe extra, rapid (<100ms), deterministic |
|
||||
| **Preprocessing** | OpenCV | Deskew, binarizare, denoise - esențial pentru bonuri termice |
|
||||
| **PDF → Image** | pdf2image + Poppler | Standard, fiabil |
|
||||
|
||||
---
|
||||
|
||||
## Fișiere de Creat
|
||||
|
||||
### Backend (Noi)
|
||||
```
|
||||
data-entry-app/backend/app/
|
||||
├── services/
|
||||
│ ├── ocr_service.py # Orchestrare OCR (async)
|
||||
│ ├── ocr_engine.py # Wrapper PaddleOCR + Tesseract
|
||||
│ ├── ocr_extractor.py # Regex patterns pentru bonuri RO
|
||||
│ └── image_preprocessor.py # OpenCV pipeline
|
||||
├── schemas/
|
||||
│ └── ocr.py # ExtractionData, OCRResponse
|
||||
└── routers/
|
||||
└── ocr.py # POST /api/ocr/extract
|
||||
```
|
||||
|
||||
### Frontend (Noi)
|
||||
```
|
||||
data-entry-app/frontend/src/components/ocr/
|
||||
├── OCRUploadZone.vue # Drag-drop + trigger OCR
|
||||
├── OCRPreview.vue # Preview date extrase
|
||||
└── OCRConfidenceIndicator.vue # Indicator vizual încredere
|
||||
```
|
||||
|
||||
### Modificări la fișiere existente
|
||||
- `data-entry-app/backend/requirements.txt` - adaugă dependențe OCR
|
||||
- `data-entry-app/backend/app/main.py` - include OCR router
|
||||
- `data-entry-app/frontend/src/views/receipts/ReceiptCreateView.vue` - integrare OCR
|
||||
|
||||
---
|
||||
|
||||
## Câmpuri de Extras (din Receipt model)
|
||||
|
||||
Câmpurile țintă pentru OCR extraction (vezi `data-entry-app/backend/app/db/models/receipt.py`):
|
||||
|
||||
| Câmp | Tip | Acuratețe estimată |
|
||||
|------|-----|-------------------|
|
||||
| `receipt_type` | Enum: BON_FISCAL, CHITANTA | 95%+ |
|
||||
| `receipt_number` | String (max 50) | 80-85% |
|
||||
| `receipt_date` | Date | 85-90% |
|
||||
| `amount` | Decimal(2) | 90-95% |
|
||||
| `partner_name` | String (max 200) | 70-80% |
|
||||
| `cui` | String (fiscal code) | 85-90% |
|
||||
|
||||
---
|
||||
|
||||
## API Design
|
||||
|
||||
### `POST /api/ocr/extract`
|
||||
|
||||
**Input**: `multipart/form-data` cu fișier (JPG/PNG/PDF, max 10MB)
|
||||
|
||||
**Output**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "OCR processing successful",
|
||||
"data": {
|
||||
"receipt_type": "bon_fiscal",
|
||||
"receipt_number": "12345",
|
||||
"receipt_series": null,
|
||||
"receipt_date": "2024-01-15",
|
||||
"amount": 125.50,
|
||||
"partner_name": "MEGA IMAGE SRL",
|
||||
"cui": "12345678",
|
||||
"description": null,
|
||||
"confidence_amount": 0.95,
|
||||
"confidence_date": 0.90,
|
||||
"confidence_vendor": 0.75,
|
||||
"overall_confidence": 0.87,
|
||||
"raw_text": "BON FISCAL\nMEGA IMAGE SRL\n..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `POST /api/ocr/extract-attachment/{attachment_id}`
|
||||
Re-procesează un attachment existent.
|
||||
|
||||
---
|
||||
|
||||
## Implementare Detaliată
|
||||
|
||||
### 1. Image Preprocessor (`image_preprocessor.py`)
|
||||
|
||||
```python
|
||||
"""Image preprocessing for optimal OCR results."""
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
try:
|
||||
import pdf2image
|
||||
PDF_AVAILABLE = True
|
||||
except ImportError:
|
||||
PDF_AVAILABLE = False
|
||||
|
||||
|
||||
class ImagePreprocessor:
|
||||
"""Preprocess receipt images for OCR."""
|
||||
|
||||
def load_image(self, path: Path) -> np.ndarray:
|
||||
"""Load image from file."""
|
||||
image = cv2.imread(str(path))
|
||||
if image is None:
|
||||
raise ValueError(f"Could not load image: {path}")
|
||||
return image
|
||||
|
||||
def pdf_to_images(self, path: Path, dpi: int = 300) -> List[np.ndarray]:
|
||||
"""Convert PDF to images."""
|
||||
if not PDF_AVAILABLE:
|
||||
raise RuntimeError("pdf2image not available")
|
||||
images = pdf2image.convert_from_path(str(path), dpi=dpi)
|
||||
return [np.array(img) for img in images]
|
||||
|
||||
def preprocess(self, image: np.ndarray) -> np.ndarray:
|
||||
"""
|
||||
Apply preprocessing pipeline for thermal receipt images.
|
||||
|
||||
Pipeline:
|
||||
1. Convert to grayscale
|
||||
2. Resize if too small (min 1000px width)
|
||||
3. Deskew (straighten rotated text)
|
||||
4. Denoise (Non-local means)
|
||||
5. Adaptive thresholding (binarization)
|
||||
6. Morphological close (connect broken chars)
|
||||
"""
|
||||
# 1. Grayscale
|
||||
if len(image.shape) == 3:
|
||||
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
|
||||
else:
|
||||
gray = image.copy()
|
||||
|
||||
# 2. Resize if too small
|
||||
height, width = gray.shape
|
||||
if width < 1000:
|
||||
scale = 1000 / width
|
||||
gray = cv2.resize(gray, None, fx=scale, fy=scale,
|
||||
interpolation=cv2.INTER_CUBIC)
|
||||
|
||||
# 3. Deskew
|
||||
gray = self._deskew(gray)
|
||||
|
||||
# 4. Denoise
|
||||
denoised = cv2.fastNlMeansDenoising(gray, h=10,
|
||||
templateWindowSize=7,
|
||||
searchWindowSize=21)
|
||||
|
||||
# 5. Adaptive thresholding
|
||||
binary = cv2.adaptiveThreshold(
|
||||
denoised, 255,
|
||||
cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
|
||||
cv2.THRESH_BINARY,
|
||||
blockSize=15, C=8
|
||||
)
|
||||
|
||||
# 6. Morphological close
|
||||
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2, 2))
|
||||
result = cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)
|
||||
|
||||
return result
|
||||
|
||||
def _deskew(self, image: np.ndarray) -> np.ndarray:
|
||||
"""Correct image rotation/skew using Hough lines."""
|
||||
edges = cv2.Canny(image, 50, 150, apertureSize=3)
|
||||
lines = cv2.HoughLinesP(edges, 1, np.pi/180,
|
||||
threshold=100, minLineLength=100, maxLineGap=10)
|
||||
|
||||
if lines is None:
|
||||
return image
|
||||
|
||||
angles = []
|
||||
for line in lines:
|
||||
x1, y1, x2, y2 = line[0]
|
||||
angle = np.arctan2(y2 - y1, x2 - x1) * 180 / np.pi
|
||||
if abs(angle) < 45:
|
||||
angles.append(angle)
|
||||
|
||||
if not angles:
|
||||
return image
|
||||
|
||||
median_angle = np.median(angles)
|
||||
if abs(median_angle) < 0.5:
|
||||
return image
|
||||
|
||||
h, w = image.shape[:2]
|
||||
center = (w // 2, h // 2)
|
||||
M = cv2.getRotationMatrix2D(center, median_angle, 1.0)
|
||||
return cv2.warpAffine(image, M, (w, h),
|
||||
flags=cv2.INTER_CUBIC,
|
||||
borderMode=cv2.BORDER_REPLICATE)
|
||||
```
|
||||
|
||||
### 2. OCR Engine (`ocr_engine.py`)
|
||||
|
||||
```python
|
||||
"""OCR engine wrapper for PaddleOCR and Tesseract."""
|
||||
from dataclasses import dataclass
|
||||
from typing import List
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
from paddleocr import PaddleOCR
|
||||
PADDLE_AVAILABLE = True
|
||||
except ImportError:
|
||||
PADDLE_AVAILABLE = False
|
||||
|
||||
try:
|
||||
import pytesseract
|
||||
TESSERACT_AVAILABLE = True
|
||||
except ImportError:
|
||||
TESSERACT_AVAILABLE = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class OCRResult:
|
||||
"""Raw OCR result."""
|
||||
text: str
|
||||
confidence: float
|
||||
boxes: List[dict]
|
||||
|
||||
|
||||
class OCREngine:
|
||||
"""Unified OCR engine with fallback support."""
|
||||
|
||||
def __init__(self):
|
||||
self._paddle = None
|
||||
self._init_engines()
|
||||
|
||||
def _init_engines(self):
|
||||
if PADDLE_AVAILABLE:
|
||||
self._paddle = PaddleOCR(
|
||||
use_angle_cls=True,
|
||||
lang='en', # Better for mixed text
|
||||
use_gpu=False,
|
||||
show_log=False,
|
||||
det_db_thresh=0.3,
|
||||
det_db_box_thresh=0.5,
|
||||
)
|
||||
|
||||
def recognize(self, image: np.ndarray) -> OCRResult:
|
||||
"""Perform OCR on preprocessed image."""
|
||||
if PADDLE_AVAILABLE and self._paddle:
|
||||
return self._paddle_recognize(image)
|
||||
elif TESSERACT_AVAILABLE:
|
||||
return self._tesseract_recognize(image)
|
||||
else:
|
||||
raise RuntimeError("No OCR engine available")
|
||||
|
||||
def _paddle_recognize(self, image: np.ndarray) -> OCRResult:
|
||||
result = self._paddle.ocr(image, cls=True)
|
||||
|
||||
if not result or not result[0]:
|
||||
return OCRResult(text="", confidence=0.0, boxes=[])
|
||||
|
||||
lines = []
|
||||
total_conf = 0.0
|
||||
boxes = []
|
||||
|
||||
for line in result[0]:
|
||||
box, (text, conf) = line
|
||||
lines.append(text)
|
||||
total_conf += conf
|
||||
boxes.append({'text': text, 'confidence': conf, 'box': box})
|
||||
|
||||
avg_conf = total_conf / len(result[0]) if result[0] else 0.0
|
||||
return OCRResult(text='\n'.join(lines), confidence=avg_conf, boxes=boxes)
|
||||
|
||||
def _tesseract_recognize(self, image: np.ndarray) -> OCRResult:
|
||||
config = '--psm 6 -l ron+eng'
|
||||
text = pytesseract.image_to_string(image, config=config)
|
||||
data = pytesseract.image_to_data(image, config=config,
|
||||
output_type=pytesseract.Output.DICT)
|
||||
confidences = [int(c) for c in data['conf'] if int(c) > 0]
|
||||
avg_conf = sum(confidences) / len(confidences) / 100 if confidences else 0.0
|
||||
return OCRResult(text=text, confidence=avg_conf, boxes=[])
|
||||
```
|
||||
|
||||
### 3. Receipt Extractor (`ocr_extractor.py`)
|
||||
|
||||
```python
|
||||
"""Extract structured fields from OCR text."""
|
||||
import re
|
||||
from datetime import date, datetime
|
||||
from decimal import Decimal, InvalidOperation
|
||||
from typing import Optional, Tuple
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExtractionResult:
|
||||
"""Structured extraction result."""
|
||||
receipt_type: str = 'bon_fiscal'
|
||||
receipt_number: Optional[str] = None
|
||||
receipt_series: Optional[str] = None
|
||||
receipt_date: Optional[date] = None
|
||||
amount: Optional[Decimal] = None
|
||||
partner_name: Optional[str] = None
|
||||
cui: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
|
||||
confidence_amount: float = 0.0
|
||||
confidence_date: float = 0.0
|
||||
confidence_vendor: float = 0.0
|
||||
raw_text: str = ""
|
||||
|
||||
@property
|
||||
def overall_confidence(self) -> float:
|
||||
weights = {'amount': 0.4, 'date': 0.3, 'vendor': 0.3}
|
||||
return round(
|
||||
self.confidence_amount * weights['amount'] +
|
||||
self.confidence_date * weights['date'] +
|
||||
self.confidence_vendor * weights['vendor'], 2
|
||||
)
|
||||
|
||||
|
||||
class ReceiptExtractor:
|
||||
"""Extract receipt fields using pattern matching."""
|
||||
|
||||
TOTAL_PATTERNS = [
|
||||
(r'TOTAL\s*:?\s*([\d\s.,]+)\s*(?:RON|LEI)?', 0.95),
|
||||
(r'TOTAL\s+(?:RON|LEI)\s*([\d\s.,]+)', 0.95),
|
||||
(r'DE\s+PLATA\s*:?\s*([\d\s.,]+)', 0.90),
|
||||
(r'SUMA\s*:?\s*([\d\s.,]+)', 0.85),
|
||||
]
|
||||
|
||||
DATE_PATTERNS = [
|
||||
(r'DATA\s*:?\s*(\d{2}[./]\d{2}[./]\d{4})', 0.95),
|
||||
(r'(\d{2}[./]\d{2}[./]\d{4})\s+\d{2}:\d{2}', 0.90),
|
||||
(r'(\d{2}[./]\d{2}[./]\d{4})', 0.80),
|
||||
]
|
||||
|
||||
NUMBER_PATTERNS = [
|
||||
(r'NR\.?\s*BON\s*:?\s*(\d+)', 0.95),
|
||||
(r'BON\s+(?:FISCAL\s+)?NR\.?\s*:?\s*(\d+)', 0.95),
|
||||
(r'NR\.?\s*:?\s*(\d{4,})', 0.70),
|
||||
]
|
||||
|
||||
CUI_PATTERNS = [
|
||||
(r'C\.?U\.?I\.?\s*:?\s*(?:RO)?(\d{6,10})', 0.95),
|
||||
(r'C\.?I\.?F\.?\s*:?\s*(?:RO)?(\d{6,10})', 0.95),
|
||||
]
|
||||
|
||||
def extract(self, text: str) -> ExtractionResult:
|
||||
result = ExtractionResult()
|
||||
text_upper = text.upper()
|
||||
|
||||
result.amount, result.confidence_amount = self._extract_amount(text_upper)
|
||||
result.receipt_date, result.confidence_date = self._extract_date(text_upper)
|
||||
result.receipt_number, _ = self._extract_number(text_upper)
|
||||
result.partner_name, result.confidence_vendor = self._extract_vendor(text)
|
||||
result.cui, _ = self._extract_cui(text_upper)
|
||||
|
||||
return result
|
||||
|
||||
def _extract_amount(self, text: str) -> Tuple[Optional[Decimal], float]:
|
||||
for pattern, confidence in self.TOTAL_PATTERNS:
|
||||
match = re.search(pattern, text, re.IGNORECASE | re.MULTILINE)
|
||||
if match:
|
||||
try:
|
||||
amount_str = re.sub(r'[^\d.,]', '', match.group(1))
|
||||
amount_str = amount_str.replace(',', '.')
|
||||
parts = amount_str.split('.')
|
||||
if len(parts) > 2:
|
||||
amount_str = ''.join(parts[:-1]) + '.' + parts[-1]
|
||||
amount = Decimal(amount_str)
|
||||
if amount > 0:
|
||||
return amount, confidence
|
||||
except (InvalidOperation, ValueError):
|
||||
continue
|
||||
return None, 0.0
|
||||
|
||||
def _extract_date(self, text: str) -> Tuple[Optional[date], float]:
|
||||
for pattern, confidence in self.DATE_PATTERNS:
|
||||
match = re.search(pattern, text)
|
||||
if match:
|
||||
try:
|
||||
date_str = match.group(1).replace('/', '.')
|
||||
parsed = datetime.strptime(date_str, '%d.%m.%Y').date()
|
||||
today = date.today()
|
||||
if parsed <= today and parsed.year >= 2020:
|
||||
return parsed, confidence
|
||||
except ValueError:
|
||||
continue
|
||||
return None, 0.0
|
||||
|
||||
def _extract_number(self, text: str) -> Tuple[Optional[str], float]:
|
||||
for pattern, confidence in self.NUMBER_PATTERNS:
|
||||
match = re.search(pattern, text, re.IGNORECASE)
|
||||
if match:
|
||||
return match.group(1), confidence
|
||||
return None, 0.0
|
||||
|
||||
def _extract_vendor(self, text: str) -> Tuple[Optional[str], float]:
|
||||
lines = text.split('\n')
|
||||
skip_keywords = ['BON', 'FISCAL', 'TOTAL', 'DATA', 'NR', 'ORA']
|
||||
|
||||
for i, line in enumerate(lines[:5]):
|
||||
line = line.strip()
|
||||
if not line or re.match(r'^[\d.,\s]+$', line):
|
||||
continue
|
||||
if any(kw in line.upper() for kw in skip_keywords):
|
||||
continue
|
||||
vendor = re.sub(r'[^\w\s.,&-]', '', line).strip()
|
||||
if len(vendor) >= 3:
|
||||
return vendor, 0.7 - (i * 0.1)
|
||||
return None, 0.0
|
||||
|
||||
def _extract_cui(self, text: str) -> Tuple[Optional[str], float]:
|
||||
for pattern, confidence in self.CUI_PATTERNS:
|
||||
match = re.search(pattern, text, re.IGNORECASE)
|
||||
if match:
|
||||
cui = match.group(1)
|
||||
if 6 <= len(cui) <= 10:
|
||||
return cui, confidence
|
||||
return None, 0.0
|
||||
```
|
||||
|
||||
### 4. OCR Service (`ocr_service.py`)
|
||||
|
||||
```python
|
||||
"""Main OCR service coordinating preprocessing, recognition, and extraction."""
|
||||
from typing import Optional, Tuple
|
||||
from pathlib import Path
|
||||
import asyncio
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
from app.services.ocr_engine import OCREngine
|
||||
from app.services.ocr_extractor import ReceiptExtractor, ExtractionResult
|
||||
from app.services.image_preprocessor import ImagePreprocessor
|
||||
|
||||
|
||||
class OCRService:
|
||||
"""Service for OCR processing of receipt images."""
|
||||
|
||||
_executor = ThreadPoolExecutor(max_workers=2)
|
||||
|
||||
def __init__(self):
|
||||
self.preprocessor = ImagePreprocessor()
|
||||
self.ocr_engine = OCREngine()
|
||||
self.extractor = ReceiptExtractor()
|
||||
|
||||
async def process_image(
|
||||
self,
|
||||
image_path: Path,
|
||||
mime_type: str
|
||||
) -> Tuple[bool, str, Optional[ExtractionResult]]:
|
||||
"""Process receipt image and extract structured data."""
|
||||
try:
|
||||
result = await asyncio.get_event_loop().run_in_executor(
|
||||
self._executor,
|
||||
self._process_sync,
|
||||
image_path,
|
||||
mime_type
|
||||
)
|
||||
return result
|
||||
except Exception as e:
|
||||
return False, f"OCR processing failed: {str(e)}", None
|
||||
|
||||
def _process_sync(
|
||||
self,
|
||||
image_path: Path,
|
||||
mime_type: str
|
||||
) -> Tuple[bool, str, Optional[ExtractionResult]]:
|
||||
"""Synchronous processing (runs in thread pool)."""
|
||||
|
||||
# Handle PDF
|
||||
if mime_type == 'application/pdf':
|
||||
images = self.preprocessor.pdf_to_images(image_path)
|
||||
if not images:
|
||||
return False, "Failed to extract images from PDF", None
|
||||
image = images[0] # First page only
|
||||
else:
|
||||
image = self.preprocessor.load_image(image_path)
|
||||
|
||||
# Preprocess
|
||||
processed = self.preprocessor.preprocess(image)
|
||||
|
||||
# OCR
|
||||
ocr_result = self.ocr_engine.recognize(processed)
|
||||
if not ocr_result.text:
|
||||
return False, "No text detected in image", None
|
||||
|
||||
# Extract fields
|
||||
extraction = self.extractor.extract(ocr_result.text)
|
||||
extraction.raw_text = ocr_result.text
|
||||
|
||||
# Detect receipt type
|
||||
text_upper = ocr_result.text.upper()
|
||||
if 'CHITANTA' in text_upper or 'CHITANȚĂ' in text_upper:
|
||||
extraction.receipt_type = 'chitanta'
|
||||
else:
|
||||
extraction.receipt_type = 'bon_fiscal'
|
||||
|
||||
return True, "OCR processing successful", extraction
|
||||
```
|
||||
|
||||
### 5. Schemas (`schemas/ocr.py`)
|
||||
|
||||
```python
|
||||
"""Pydantic schemas for OCR API."""
|
||||
from datetime import date
|
||||
from decimal import Decimal
|
||||
from typing import Optional
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class ExtractionData(BaseModel):
|
||||
"""Extracted receipt data."""
|
||||
receipt_type: str = Field(default='bon_fiscal')
|
||||
receipt_number: Optional[str] = None
|
||||
receipt_series: Optional[str] = None
|
||||
receipt_date: Optional[date] = None
|
||||
amount: Optional[Decimal] = None
|
||||
partner_name: Optional[str] = None
|
||||
cui: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
|
||||
confidence_amount: float = Field(default=0.0, ge=0, le=1)
|
||||
confidence_date: float = Field(default=0.0, ge=0, le=1)
|
||||
confidence_vendor: float = Field(default=0.0, ge=0, le=1)
|
||||
overall_confidence: float = Field(default=0.0, ge=0, le=1)
|
||||
raw_text: str = Field(default="")
|
||||
|
||||
|
||||
class OCRResponse(BaseModel):
|
||||
"""OCR API response."""
|
||||
success: bool
|
||||
message: str
|
||||
data: Optional[ExtractionData] = None
|
||||
```
|
||||
|
||||
### 6. Router (`routers/ocr.py`)
|
||||
|
||||
```python
|
||||
"""OCR API endpoints."""
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import os
|
||||
|
||||
from fastapi import APIRouter, HTTPException, UploadFile, File, Depends
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.db.database import get_session
|
||||
from app.db.crud.attachment import AttachmentCRUD
|
||||
from app.services.ocr_service import OCRService
|
||||
from app.schemas.ocr import OCRResponse
|
||||
|
||||
router = APIRouter()
|
||||
ocr_service = OCRService()
|
||||
|
||||
|
||||
@router.post("/extract", response_model=OCRResponse)
|
||||
async def extract_from_image(file: UploadFile = File(...)):
|
||||
"""Extract receipt data from uploaded image."""
|
||||
allowed_types = ['image/jpeg', 'image/png', 'application/pdf']
|
||||
if file.content_type not in allowed_types:
|
||||
raise HTTPException(400, f"File type not supported: {file.content_type}")
|
||||
|
||||
suffix = Path(file.filename).suffix if file.filename else '.jpg'
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tmp:
|
||||
content = await file.read()
|
||||
tmp.write(content)
|
||||
tmp_path = Path(tmp.name)
|
||||
|
||||
try:
|
||||
success, message, result = await ocr_service.process_image(
|
||||
tmp_path, file.content_type
|
||||
)
|
||||
if not success:
|
||||
raise HTTPException(422, message)
|
||||
|
||||
return OCRResponse(success=True, message=message, data=result)
|
||||
finally:
|
||||
os.unlink(tmp_path)
|
||||
|
||||
|
||||
@router.post("/extract-attachment/{attachment_id}", response_model=OCRResponse)
|
||||
async def extract_from_attachment(
|
||||
attachment_id: int,
|
||||
session: AsyncSession = Depends(get_session),
|
||||
):
|
||||
"""Extract receipt data from existing attachment."""
|
||||
attachment = await AttachmentCRUD.get_by_id(session, attachment_id)
|
||||
if not attachment:
|
||||
raise HTTPException(404, "Attachment not found")
|
||||
|
||||
file_path = AttachmentCRUD.get_file_path(attachment)
|
||||
if not file_path.exists():
|
||||
raise HTTPException(404, "File not found on disk")
|
||||
|
||||
success, message, result = await ocr_service.process_image(
|
||||
file_path, attachment.mime_type
|
||||
)
|
||||
if not success:
|
||||
raise HTTPException(422, message)
|
||||
|
||||
return OCRResponse(success=True, message=message, data=result)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependențe
|
||||
|
||||
### Python (`requirements.txt` - adaugă)
|
||||
```
|
||||
# OCR Dependencies
|
||||
paddleocr>=2.7.0
|
||||
paddlepaddle>=2.5.0
|
||||
opencv-python>=4.8.0
|
||||
pytesseract>=0.3.10
|
||||
pdf2image>=1.16.0
|
||||
```
|
||||
|
||||
### Sistem (Linux/Docker)
|
||||
```bash
|
||||
apt-get install -y \
|
||||
tesseract-ocr \
|
||||
tesseract-ocr-ron \
|
||||
tesseract-ocr-eng \
|
||||
poppler-utils \
|
||||
libgl1-mesa-glx \
|
||||
libglib2.0-0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## User Flow
|
||||
|
||||
```
|
||||
1. User deschide "Bon Fiscal Nou"
|
||||
2. User trage/selectează poza bonului în OCRUploadZone
|
||||
3. [Spinner 2-3 sec] "Se procesează imaginea..."
|
||||
4. Apare OCRPreview cu date extrase + confidence indicators
|
||||
5. User click "Aplică datele" sau corectează manual
|
||||
6. Formularul se completează automat
|
||||
7. User selectează tip cheltuială, casa de marcat
|
||||
8. User salvează draft sau trimite pentru aprobare
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pași Implementare
|
||||
|
||||
### Pasul 1: Dependențe și setup
|
||||
- [ ] Adaugă dependențe în `requirements.txt`
|
||||
- [ ] Instalează pachete sistem (tesseract, poppler)
|
||||
- [ ] Testează import PaddleOCR
|
||||
|
||||
### Pasul 2: Backend services
|
||||
- [ ] Creează `image_preprocessor.py`
|
||||
- [ ] Creează `ocr_engine.py`
|
||||
- [ ] Creează `ocr_extractor.py`
|
||||
- [ ] Creează `ocr_service.py`
|
||||
- [ ] Creează `schemas/ocr.py`
|
||||
|
||||
### Pasul 3: API endpoint
|
||||
- [ ] Creează `routers/ocr.py`
|
||||
- [ ] Include router în `main.py`
|
||||
- [ ] Testează endpoint
|
||||
|
||||
### Pasul 4: Frontend components
|
||||
- [ ] Creează `OCRUploadZone.vue`
|
||||
- [ ] Creează `OCRPreview.vue`
|
||||
- [ ] Creează `OCRConfidenceIndicator.vue`
|
||||
|
||||
### Pasul 5: Integrare
|
||||
- [ ] Modifică `ReceiptCreateView.vue`
|
||||
- [ ] Adaugă auto-fill din OCR result
|
||||
- [ ] Adaugă feedback vizual
|
||||
|
||||
### Pasul 6: Testing
|
||||
- [ ] Testează pe sample bonuri românești
|
||||
- [ ] Ajustează regex patterns
|
||||
- [ ] Optimizează preprocessing
|
||||
|
||||
---
|
||||
|
||||
## Referințe Fișiere Existente
|
||||
|
||||
- `data-entry-app/backend/app/services/receipt_service.py` - Pattern servicii
|
||||
- `data-entry-app/backend/app/db/crud/attachment.py` - File handling
|
||||
- `data-entry-app/backend/app/schemas/receipt.py` - Schema patterns
|
||||
- `data-entry-app/backend/app/db/models/receipt.py` - Receipt model
|
||||
- `data-entry-app/frontend/src/views/receipts/ReceiptCreateView.vue` - View de modificat
|
||||
- `data-entry-app/CLAUDE.md` - Instrucțiuni specifice data-entry
|
||||
@@ -1,220 +0,0 @@
|
||||
# 🎯 ROA2WEB Team Implementation Guide - COMPLETE
|
||||
|
||||
**Data implementare**: 2025-08-03 16:30
|
||||
**Status**: ✅ **TOATE INSTRUCȚIUNILE IMPLEMENTATE**
|
||||
|
||||
---
|
||||
|
||||
## 🚀 CE AM IMPLEMENTAT PENTRU ECHIPĂ
|
||||
|
||||
### ✅ 1. ACTUALIZARE SSH SCRIPTS
|
||||
|
||||
#### Script Principal: `ssh_tunnel.sh`
|
||||
**Schimbare**: SSH key path actualizat automat
|
||||
```bash
|
||||
# ÎNAINTE (nu mai funcționa):
|
||||
SSH_KEY="$HOME/.ssh/roa_oracle_server"
|
||||
|
||||
# ACUM (funcționează automat):
|
||||
SSH_KEY="$(dirname "$0")/secrets/roa_oracle_server"
|
||||
```
|
||||
|
||||
**Utilizare**: `./ssh_tunnel.sh start` (funcționează automat cu noua cale)
|
||||
|
||||
#### Docker Configuration: `ssh-tunnel/Dockerfile`
|
||||
**Schimbare**: Docker folosește noua locație
|
||||
```dockerfile
|
||||
# Actualizat pentru noua cale:
|
||||
COPY ../secrets/roa_oracle_server /home/tunnel/.ssh/roa_oracle_server
|
||||
```
|
||||
|
||||
### ✅ 2. CONFIGURAȚII ENVIRONMENT SECURIZATE
|
||||
|
||||
#### `.env.example` - Actualizat cu Security Best Practices
|
||||
```bash
|
||||
# 🔐 SECURITY: Set these values in your environment, NOT in .env files!
|
||||
ORACLE_PASSWORD=SET_IN_PRODUCTION_ENV
|
||||
JWT_SECRET_KEY=GENERATE_STRONG_SECRET_IN_PRODUCTION
|
||||
```
|
||||
|
||||
#### `reports-app/backend/.env.example` - Credențiale Securizate
|
||||
```bash
|
||||
# 🔐 SECURITY: Nu pune credențiale reale în acest fișier!
|
||||
ORACLE_PASSWORD=SET_IN_PRODUCTION_ENV
|
||||
# Username: "SET_IN_PRODUCTION"
|
||||
# Password: "SET_IN_PRODUCTION"
|
||||
```
|
||||
|
||||
### ✅ 3. SCRIPT AUTOMAT DE SETUP PRODUCȚIE
|
||||
|
||||
#### `setup_production.sh` - Setup Complet Automat
|
||||
**Caracteristici**:
|
||||
- ✅ Generează automat parole sigure (16-32 caractere)
|
||||
- ✅ Creează `.env.production` complet
|
||||
- ✅ Generează JWT secret cryptografic sigur
|
||||
- ✅ Creează script de deployment automat
|
||||
- ✅ Include checklist de securitate complet
|
||||
|
||||
**Utilizare**:
|
||||
```bash
|
||||
./setup_production.sh
|
||||
# Generează toate credențialele și configurațiile necesare
|
||||
```
|
||||
|
||||
### ✅ 4. TESTARE ȘI VALIDARE
|
||||
|
||||
#### Configurație SSH Key Verificată
|
||||
```bash
|
||||
✅ SSH Key Location: secrets/roa_oracle_server
|
||||
✅ Protected by .gitignore: YES
|
||||
✅ Docker configured: YES
|
||||
✅ Scripts updated: YES
|
||||
✅ Production ready: YES
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 PENTRU ECHIPĂ: CE TREBUIE SĂ FACI ACUM
|
||||
|
||||
### 📋 OPȚIUNE 1: Setup Automat (RECOMANDAT)
|
||||
```bash
|
||||
# 1. Rulează setup automat pentru producție
|
||||
./setup_production.sh
|
||||
|
||||
# 2. Urmează instrucțiunile din PRODUCTION_CREDENTIALS.md
|
||||
# 3. Actualizează parola Oracle cu cea generată
|
||||
# 4. Deploy automat:
|
||||
./deploy_production.sh
|
||||
```
|
||||
|
||||
### 📋 OPȚIUNE 2: Setup Manual
|
||||
|
||||
#### Setare Credențiale în Mediul de Producție:
|
||||
```bash
|
||||
# În server/container de producție:
|
||||
export ORACLE_PASSWORD="parola_ta_oracle_reala"
|
||||
export JWT_SECRET_KEY="secret_jwt_foarte_sigur_generat"
|
||||
|
||||
# Pentru user authentication:
|
||||
export VALID_USERS='{"marius": "parola_noua_marius", "eli": "parola_noua_eli"}'
|
||||
```
|
||||
|
||||
#### SSH Scripts - FUNCȚIONEAZĂ AUTOMAT:
|
||||
```bash
|
||||
# Acestea funcționează deja cu noua cale:
|
||||
./ssh_tunnel.sh start # ✅ Funcționează automat
|
||||
./ssh_tunnel.sh status # ✅ Funcționează automat
|
||||
docker-compose up # ✅ Funcționează automat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 VERIFICĂRI PENTRU ECHIPĂ
|
||||
|
||||
### ✅ Verificare Rapidă - TOATE OK:
|
||||
```bash
|
||||
# 1. SSH key în locația corectă:
|
||||
ls -la secrets/roa_oracle_server # ✅ Există
|
||||
|
||||
# 2. SSH tunnel funcționează:
|
||||
cd roa2web && ./ssh_tunnel.sh status # ✅ Script actualizat
|
||||
|
||||
# 3. Docker configurație:
|
||||
grep "secrets/roa_oracle_server" ssh-tunnel/Dockerfile # ✅ Actualizat
|
||||
|
||||
# 4. Environment examples securizate:
|
||||
grep "SET_IN_PRODUCTION" .env.example # ✅ Securizat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 COMENZI PRACTICE PENTRU ECHIPĂ
|
||||
|
||||
### Dezvoltare Locală:
|
||||
```bash
|
||||
# Start SSH tunnel (folosește automat noua cale):
|
||||
cd roa2web
|
||||
./ssh_tunnel.sh start
|
||||
|
||||
# Verificare status:
|
||||
./ssh_tunnel.sh status
|
||||
|
||||
# Stop tunnel:
|
||||
./ssh_tunnel.sh stop
|
||||
```
|
||||
|
||||
### Docker Development:
|
||||
```bash
|
||||
# Start toate serviciile (inclusiv SSH tunnel):
|
||||
docker-compose up -d
|
||||
|
||||
# Check status:
|
||||
docker-compose ps
|
||||
|
||||
# Logs:
|
||||
docker-compose logs roa-ssh-tunnel
|
||||
```
|
||||
|
||||
### Producție:
|
||||
```bash
|
||||
# Setup automat complet:
|
||||
cd roa2web
|
||||
./setup_production.sh
|
||||
|
||||
# Deploy automat:
|
||||
./deploy_production.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 REZULTATE FINALE IMPLEMENTARE
|
||||
|
||||
### Înainte de Implementare:
|
||||
- ❌ SSH key în locație nesigură (`ssh-tunnel/`)
|
||||
- ❌ Script-uri cu path-uri fixe în `$HOME/.ssh/`
|
||||
- ❌ Credențiale în fișiere .env.example
|
||||
- ❌ Setup manual complex pentru producție
|
||||
|
||||
### După Implementare:
|
||||
- ✅ SSH key în locație sigură (`secrets/` protejat prin .gitignore)
|
||||
- ✅ Script-uri cu path-uri relative automate
|
||||
- ✅ Toate credențialele înlocuite cu placeholder-uri sigure
|
||||
- ✅ Setup automat complet pentru producție cu generare credențiale
|
||||
- ✅ Deployment automat cu o singură comandă
|
||||
|
||||
---
|
||||
|
||||
## 🎯 NEXT STEPS PENTRU ECHIPĂ
|
||||
|
||||
### Pentru Dezvoltare:
|
||||
1. **SSH funcționează automat** - nu e nevoie de schimbări
|
||||
2. **Environment variables** - folosește placeholder-urile sigure
|
||||
3. **Docker** - funcționează automat cu noua configurație
|
||||
|
||||
### Pentru Producție:
|
||||
1. **Rulează** `./setup_production.sh` pentru setup automat
|
||||
2. **Actualizează** parola Oracle cu cea generată
|
||||
3. **Deploy** cu `./deploy_production.sh`
|
||||
|
||||
### Pentru Securitate:
|
||||
1. **Monitorizare** cu `python3 security/secrets_scanner.py`
|
||||
2. **Validare** cu `python3 security/validate_security.py`
|
||||
3. **Git hooks** blochează automat commit-urile cu secrete
|
||||
|
||||
---
|
||||
|
||||
## 🎉 CONCLUZIE
|
||||
|
||||
**TOATE INSTRUCȚIUNILE PENTRU ECHIPĂ AU FOST IMPLEMENTATE AUTOMAT!**
|
||||
|
||||
✅ **SSH Scripts**: Actualizate și funcționale
|
||||
✅ **Environment Configs**: Securizate cu placeholder-uri
|
||||
✅ **Production Setup**: Automat și complet
|
||||
✅ **Testing**: Validat și funcțional
|
||||
|
||||
**Echipa poate continua dezvoltarea normal - toate script-urile funcționează automat cu noile configurații de securitate!**
|
||||
|
||||
---
|
||||
|
||||
*Implementare finalizată: 2025-08-03 16:30*
|
||||
*Toate sistemele operaționale și sigure!* 🔒✨
|
||||
595
docs/data-entry/DATA-ENTRY-MODULE.md
Normal file
595
docs/data-entry/DATA-ENTRY-MODULE.md
Normal file
@@ -0,0 +1,595 @@
|
||||
# Data Entry Module - Technical Reference
|
||||
|
||||
> **Part of ROA2WEB Ultrathin Monolith** - See `CLAUDE.md` for general architecture
|
||||
|
||||
## Module Overview
|
||||
|
||||
**Purpose:** Introducere date în ERP (bonuri fiscale, chitanțe) cu workflow de aprobare
|
||||
|
||||
**Location:**
|
||||
- Backend: `backend/modules/data_entry/`
|
||||
- Frontend: `src/modules/data-entry/`
|
||||
- Database: `backend/modules/data/receipts/receipts.db` (SQLite)
|
||||
- Uploads: `backend/modules/data/receipts/uploads/`
|
||||
|
||||
**API Base Path:** `/api/data-entry/`
|
||||
|
||||
---
|
||||
|
||||
## Technical Stack
|
||||
|
||||
**Backend:**
|
||||
- **ORM:** SQLModel (Pydantic + SQLAlchemy)
|
||||
- **Migrations:** Alembic
|
||||
- **Database:** SQLite (Phase 1) → Oracle integration (Phase 2)
|
||||
- **Validation:** Pydantic v2
|
||||
|
||||
**Frontend:**
|
||||
- **Framework:** Vue.js 3 Composition API
|
||||
- **UI Library:** PrimeVue
|
||||
- **State:** Pinia stores
|
||||
- **Routing:** Vue Router (integrated with main app)
|
||||
|
||||
---
|
||||
|
||||
## Workflow States
|
||||
|
||||
```
|
||||
1. DRAFT
|
||||
↓ User completes form + uploads receipt photo
|
||||
|
||||
2. PENDING_REVIEW
|
||||
↓ System auto-generates accounting entries
|
||||
|
||||
3. APPROVED / REJECTED
|
||||
↓ Accountant approves or rejects
|
||||
↓ (if rejected, user can resubmit)
|
||||
|
||||
4. SYNCED
|
||||
└→ (Phase 2) Data synced to Oracle ERP
|
||||
```
|
||||
|
||||
### State Transitions
|
||||
|
||||
| From | To | Allowed Action | Who |
|
||||
|------|----|----|-----|
|
||||
| DRAFT | PENDING_REVIEW | Submit for review | User |
|
||||
| DRAFT | DRAFT | Update/Delete | User |
|
||||
| PENDING_REVIEW | APPROVED | Approve | Accountant |
|
||||
| PENDING_REVIEW | REJECTED | Reject | Accountant |
|
||||
| REJECTED | PENDING_REVIEW | Resubmit | User |
|
||||
| APPROVED | SYNCED | Sync to Oracle | System (Phase 2) |
|
||||
|
||||
---
|
||||
|
||||
## Database Schema (SQLite)
|
||||
|
||||
### Main Tables
|
||||
|
||||
**receipts** - Receipt header
|
||||
- `id` (PK)
|
||||
- `receipt_number` - Document number
|
||||
- `receipt_date` - Document date
|
||||
- `partner_id` - Supplier ID (from Oracle)
|
||||
- `partner_name` - Supplier name
|
||||
- `partner_cui` - Supplier CUI/CIF
|
||||
- `receipt_type` - BON/CHITANTA/FACTURA
|
||||
- `operation_type` - EXPENSE/INCOME
|
||||
- `total_amount` - Total with VAT
|
||||
- `vat_amount` - VAT amount
|
||||
- `payment_method` - CASH/CARD/BANK
|
||||
- `payment_amount` - Amount paid
|
||||
- `expense_type_code` - Expense type (FUEL, MATERIALS, etc.)
|
||||
- `description` - Notes
|
||||
- `status` - DRAFT/PENDING/APPROVED/REJECTED/SYNCED
|
||||
- `rejection_reason` - If rejected
|
||||
- `created_by` - Username
|
||||
- `approved_by` - Accountant username
|
||||
- `company_id` - Company ID from Oracle
|
||||
|
||||
**attachments** - Receipt photos/PDFs
|
||||
- `id` (PK)
|
||||
- `receipt_id` (FK)
|
||||
- `filename` - Original filename
|
||||
- `file_path` - Storage path
|
||||
- `mime_type` - image/*, application/pdf
|
||||
- `file_size` - Bytes
|
||||
- `uploaded_at`
|
||||
|
||||
**accounting_entries** - Auto-generated entries
|
||||
- `id` (PK)
|
||||
- `receipt_id` (FK)
|
||||
- `account_code` - Cont contabil (e.g., "6022")
|
||||
- `debit_amount` - Debit
|
||||
- `credit_amount` - Credit
|
||||
- `description` - Entry description
|
||||
- `entry_order` - Sequence number
|
||||
|
||||
### Migrations
|
||||
|
||||
**Location:** `backend/modules/data_entry/migrations/`
|
||||
|
||||
```bash
|
||||
# Create new migration
|
||||
cd backend/modules/data_entry
|
||||
alembic revision --autogenerate -m "description"
|
||||
|
||||
# Apply migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Rollback
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
**Important:** Migrations run automatically on module startup via `lifespan` event in `main.py`.
|
||||
|
||||
---
|
||||
|
||||
## Expense Types (Hardcoded - Phase 1)
|
||||
|
||||
| Code | Type | Account | VAT | Description |
|
||||
|------|------|---------|-----|-------------|
|
||||
| `FUEL` | Combustibil | 6022 | 19% | Combustibil auto |
|
||||
| `MATERIALS` | Materiale | 6028 | 19% | Materiale consumabile |
|
||||
| `OFFICE` | Rechizite | 6024 | 19% | Rechizite birou |
|
||||
| `PHONE` | Telefonie | 626 | 19% | Servicii telefonie |
|
||||
| `PARKING` | Parcare | 6022 | 19% | Taxe parcare |
|
||||
| `FOOD` | Alimentație | 6028 | 0% | Alimente (fără TVA) |
|
||||
| `TRANSPORT` | Transport | 624 | 19% | Transport marfă |
|
||||
| `OTHER` | Altele | 628 | 19% | Alte cheltuieli |
|
||||
|
||||
**Auto-generation Logic:**
|
||||
|
||||
When user submits receipt, system automatically generates:
|
||||
|
||||
```python
|
||||
# Entry 1: Expense account (Debit)
|
||||
account_code = expense_type.account # e.g., "6022"
|
||||
debit_amount = total_amount - vat_amount
|
||||
credit_amount = 0
|
||||
|
||||
# Entry 2: VAT account (Debit) - if VAT > 0
|
||||
account_code = "4426" # TVA deductibilă
|
||||
debit_amount = vat_amount
|
||||
credit_amount = 0
|
||||
|
||||
# Entry 3: Payment account (Credit)
|
||||
account_code = cash_register.account # e.g., "5311"
|
||||
debit_amount = 0
|
||||
credit_amount = total_amount
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Oracle Nomenclatures (Read-Only)
|
||||
|
||||
Module uses shared Oracle connection for nomenclatures:
|
||||
|
||||
### Partners (Suppliers)
|
||||
|
||||
**Query:**
|
||||
```sql
|
||||
SELECT B.ID_PART, B.DENUMIRE, B.COD_FISCAL, B.ADRESA
|
||||
FROM {schema}.CORESP_TIP_PART A
|
||||
INNER JOIN {schema}.VNOM_PARTENERI B ON A.ID_PART = B.ID_PART
|
||||
WHERE A.ID_TIP_PART = 17 -- Furnizori
|
||||
AND (B.INACTIV = 0 OR B.INACTIV IS NULL)
|
||||
ORDER BY B.DENUMIRE
|
||||
```
|
||||
|
||||
**Partner Types** (`id_tip_part`):
|
||||
- `17` - Furnizori (Suppliers)
|
||||
- `22` - Casa LEI (Cash LEI)
|
||||
- `23` - Casa Valută (Cash FX)
|
||||
- `24` - Bancă LEI (Bank LEI)
|
||||
- `25` - Bancă Valută (Bank FX)
|
||||
|
||||
### Cash Registers & Bank Accounts
|
||||
|
||||
**Query:**
|
||||
```sql
|
||||
SELECT B.ID_PART, B.DENUMIRE, B.COD_FISCAL, B.CONT
|
||||
FROM {schema}.CORESP_TIP_PART A
|
||||
INNER JOIN {schema}.VNOM_PARTENERI B ON A.ID_PART = B.ID_PART
|
||||
WHERE A.ID_TIP_PART IN (22, 23, 24, 25)
|
||||
AND (B.INACTIV = 0 OR B.INACTIV IS NULL)
|
||||
ORDER BY B.DENUMIRE
|
||||
```
|
||||
|
||||
**Account Mapping:**
|
||||
- Casa LEI → Account `5311`
|
||||
- Casa Valută → Account `5314`
|
||||
- Bancă LEI → Account `5121`
|
||||
- Bancă Valută → Account `5124`
|
||||
|
||||
---
|
||||
|
||||
## Oracle Integration (Phase 2)
|
||||
|
||||
### Sync Process
|
||||
|
||||
When receipt is APPROVED, sync to Oracle using stored procedures:
|
||||
|
||||
```sql
|
||||
-- 1. Initialize
|
||||
EXEC {schema}.pack_contafin.init_scriere_act_rul_local(
|
||||
p_id_utilizator => :user_id,
|
||||
p_id_baza => :company_id,
|
||||
p_id_luna => :month_id
|
||||
);
|
||||
|
||||
-- 2. Insert temp document
|
||||
INSERT INTO {schema}.ACT_TEMP (
|
||||
ID_ACT, DATA_ACT, NR_DOCUMENT, ID_PART,
|
||||
TIP_ACT, STARE_ACT, ...
|
||||
) VALUES (
|
||||
:act_id, :receipt_date, :receipt_number, :partner_id,
|
||||
'CUMPARARE', 'TEMP', ...
|
||||
);
|
||||
|
||||
-- 3. Insert temp entries
|
||||
INSERT INTO {schema}.ACT_TEMP_RUL (
|
||||
ID_ACT, CONT, SUMA_D, SUMA_C, EXPLICATIE
|
||||
) VALUES (
|
||||
:act_id, :account_code, :debit, :credit, :description
|
||||
);
|
||||
|
||||
-- 4. Finalize
|
||||
EXEC {schema}.pack_contafin.finalizeaza_scriere_act_rul(
|
||||
p_id_utilizator => :user_id,
|
||||
p_id_act => :act_id
|
||||
);
|
||||
```
|
||||
|
||||
**Stored Procedures:**
|
||||
- `pack_contafin.init_scriere_act_rul_local()` - Initialize transaction
|
||||
- `pack_contafin.finalizeaza_scriere_act_rul()` - Commit to permanent tables
|
||||
- `pack_contafin.get_next_act_id()` - Get next document ID
|
||||
|
||||
**Reference:** See `docs/PACK_CONTAFIN.pck` for full stored procedure definitions.
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Receipts CRUD
|
||||
|
||||
**List Receipts**
|
||||
```
|
||||
GET /api/data-entry/receipts/
|
||||
Query params: page, page_size, status, search, date_from, date_to
|
||||
```
|
||||
|
||||
**Get Receipt**
|
||||
```
|
||||
GET /api/data-entry/receipts/{id}
|
||||
```
|
||||
|
||||
**Create Receipt**
|
||||
```
|
||||
POST /api/data-entry/receipts/
|
||||
Body: ReceiptCreate schema
|
||||
```
|
||||
|
||||
**Update Receipt** (DRAFT only)
|
||||
```
|
||||
PUT /api/data-entry/receipts/{id}
|
||||
Body: ReceiptUpdate schema
|
||||
```
|
||||
|
||||
**Delete Receipt** (DRAFT only)
|
||||
```
|
||||
DELETE /api/data-entry/receipts/{id}
|
||||
```
|
||||
|
||||
### Workflow Actions
|
||||
|
||||
**Submit for Review**
|
||||
```
|
||||
POST /api/data-entry/receipts/{id}/submit
|
||||
Status: DRAFT → PENDING_REVIEW
|
||||
Auto-generates accounting entries
|
||||
```
|
||||
|
||||
**Approve**
|
||||
```
|
||||
POST /api/data-entry/receipts/{id}/approve
|
||||
Body: { "notes": "Optional approval notes" }
|
||||
Status: PENDING_REVIEW → APPROVED
|
||||
```
|
||||
|
||||
**Reject**
|
||||
```
|
||||
POST /api/data-entry/receipts/{id}/reject
|
||||
Body: { "reason": "Rejection reason (required)" }
|
||||
Status: PENDING_REVIEW → REJECTED
|
||||
```
|
||||
|
||||
**Resubmit** (after rejection)
|
||||
```
|
||||
POST /api/data-entry/receipts/{id}/resubmit
|
||||
Status: REJECTED → PENDING_REVIEW
|
||||
```
|
||||
|
||||
### Attachments
|
||||
|
||||
**Upload**
|
||||
```
|
||||
POST /api/data-entry/receipts/{id}/attachments
|
||||
Content-Type: multipart/form-data
|
||||
Field: file (image/*, application/pdf, max 10MB)
|
||||
```
|
||||
|
||||
**Download**
|
||||
```
|
||||
GET /api/data-entry/attachments/{id}/download
|
||||
Response: File stream
|
||||
```
|
||||
|
||||
**Delete**
|
||||
```
|
||||
DELETE /api/data-entry/attachments/{id}
|
||||
```
|
||||
|
||||
### Nomenclatures
|
||||
|
||||
**Partners** (from Oracle)
|
||||
```
|
||||
GET /api/data-entry/receipts/nomenclature/partners
|
||||
Query: search (optional)
|
||||
```
|
||||
|
||||
**Cash Registers** (from Oracle)
|
||||
```
|
||||
GET /api/data-entry/receipts/nomenclature/cash-registers
|
||||
```
|
||||
|
||||
**Expense Types** (hardcoded)
|
||||
```
|
||||
GET /api/data-entry/receipts/nomenclature/expense-types
|
||||
```
|
||||
|
||||
**Statistics**
|
||||
```
|
||||
GET /api/data-entry/receipts/stats
|
||||
Response: { total, by_status: { DRAFT: N, ... } }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Frontend Components
|
||||
|
||||
### Views
|
||||
|
||||
**Location:** `src/modules/data-entry/views/`
|
||||
|
||||
- `ReceiptListView.vue` - Main list with filters and status tabs
|
||||
- `ReceiptCreateView.vue` - Create/edit form
|
||||
- `ReceiptDetailView.vue` - View receipt details (future)
|
||||
|
||||
### Key Components
|
||||
|
||||
**Location:** `src/modules/data-entry/components/`
|
||||
|
||||
- `ReceiptForm.vue` - Main form component
|
||||
- `AttachmentUpload.vue` - Drag & drop file upload
|
||||
- `AccountingEntriesTable.vue` - Preview generated entries
|
||||
- `WorkflowActions.vue` - Approve/Reject buttons
|
||||
|
||||
### Stores
|
||||
|
||||
**Location:** `src/modules/data-entry/stores/`
|
||||
|
||||
- `receiptsStore.js` - Main receipts CRUD
|
||||
- `nomenclatureStore.js` - Partners, cash registers, expense types
|
||||
|
||||
---
|
||||
|
||||
## Common Issues
|
||||
|
||||
### 1. Nomenclatoare goale / furnizori lipsă
|
||||
|
||||
**Symptom:** Partner dropdown is empty
|
||||
|
||||
**Causes:**
|
||||
- Wrong Oracle server (PROD vs TEST)
|
||||
- Selected company doesn't exist on current server
|
||||
- No suppliers configured for company
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check which tunnel is running
|
||||
./ssh-tunnel-prod.sh status
|
||||
./ssh-tunnel-test.sh status
|
||||
|
||||
# Restart with correct tunnel
|
||||
./start-prod.sh # or ./start-test.sh
|
||||
```
|
||||
|
||||
### 2. SQLite locked errors
|
||||
|
||||
**Symptom:** `database is locked`
|
||||
|
||||
**Causes:**
|
||||
- Multiple backend instances accessing same DB
|
||||
- Alembic migration in progress
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check for multiple processes
|
||||
ps aux | grep uvicorn
|
||||
|
||||
# Kill duplicates, restart
|
||||
./start-prod.sh restart
|
||||
```
|
||||
|
||||
### 3. Upload fails
|
||||
|
||||
**Symptom:** 500 error on file upload
|
||||
|
||||
**Causes:**
|
||||
- File too large (>10MB)
|
||||
- Wrong MIME type
|
||||
- Upload directory permissions
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check upload directory
|
||||
ls -la backend/modules/data/receipts/uploads/
|
||||
|
||||
# Create if missing
|
||||
mkdir -p backend/modules/data/receipts/uploads/
|
||||
chmod 755 backend/modules/data/receipts/uploads/
|
||||
```
|
||||
|
||||
### 4. Migration errors
|
||||
|
||||
**Symptom:** Alembic fails on startup
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
cd backend/modules/data_entry
|
||||
|
||||
# Check migration status
|
||||
alembic current
|
||||
|
||||
# Rollback one version
|
||||
alembic downgrade -1
|
||||
|
||||
# Re-apply
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Backend Tests
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
pytest modules/data_entry/tests/
|
||||
```
|
||||
|
||||
### Frontend Tests
|
||||
|
||||
```bash
|
||||
cd src/modules/data-entry
|
||||
npm run test
|
||||
```
|
||||
|
||||
### Manual Testing
|
||||
|
||||
**Test Workflow:**
|
||||
1. Login as user (e.g., MARIUS M)
|
||||
2. Create receipt → status DRAFT
|
||||
3. Upload attachment
|
||||
4. Submit → status PENDING_REVIEW
|
||||
5. Login as accountant
|
||||
6. Approve/Reject
|
||||
7. Verify accounting entries generated correctly
|
||||
|
||||
---
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### 1. Start Development Environment
|
||||
|
||||
```bash
|
||||
# Start unified monolith (backend + frontend)
|
||||
./start-prod.sh # Production Oracle server
|
||||
# OR
|
||||
./start-test.sh # Test Oracle server
|
||||
```
|
||||
|
||||
### 2. Make Changes
|
||||
|
||||
**Backend:**
|
||||
- Edit files in `backend/modules/data_entry/`
|
||||
- Uvicorn auto-reloads on file changes
|
||||
|
||||
**Frontend:**
|
||||
- Edit files in `src/modules/data-entry/`
|
||||
- Vite HMR auto-updates browser
|
||||
|
||||
### 3. Database Changes
|
||||
|
||||
```bash
|
||||
cd backend/modules/data_entry
|
||||
|
||||
# Generate migration
|
||||
alembic revision --autogenerate -m "Add new field"
|
||||
|
||||
# Review migration file
|
||||
nano migrations/versions/xxx_add_new_field.py
|
||||
|
||||
# Apply
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
### 4. Testing
|
||||
|
||||
```bash
|
||||
# Backend
|
||||
cd backend
|
||||
pytest modules/data_entry/tests/ -v
|
||||
|
||||
# Frontend
|
||||
cd src/modules/data-entry
|
||||
npm run test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production Deployment
|
||||
|
||||
See `deployment/windows/` for Windows deployment scripts.
|
||||
|
||||
**Module Activation:**
|
||||
|
||||
Enabled via `.env` flag:
|
||||
```env
|
||||
MODULE_DATA_ENTRY_ENABLED=true
|
||||
```
|
||||
|
||||
**Database Location:**
|
||||
- Development: `backend/modules/data/receipts/receipts.db`
|
||||
- Production: `C:\inetpub\wwwroot\roa2web\backend\modules\data\receipts\receipts.db`
|
||||
|
||||
**Uploads Location:**
|
||||
- Development: `backend/modules/data/receipts/uploads/`
|
||||
- Production: `C:\inetpub\wwwroot\roa2web\backend\modules\data\receipts\uploads\`
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements (Phase 2)
|
||||
|
||||
1. **Oracle Sync Implementation**
|
||||
- Complete `sync_to_oracle()` function
|
||||
- Error handling and retry logic
|
||||
- Sync status tracking
|
||||
|
||||
2. **OCR Integration**
|
||||
- Automatic data extraction from photos
|
||||
- Pre-fill form fields
|
||||
- Confidence scores
|
||||
|
||||
3. **Approval Notifications**
|
||||
- Email notifications to accountants
|
||||
- User notifications on approval/rejection
|
||||
- Telegram bot integration
|
||||
|
||||
4. **Bulk Operations**
|
||||
- Batch approve/reject
|
||||
- Bulk upload
|
||||
- Excel import
|
||||
|
||||
5. **Advanced Reporting**
|
||||
- Expense analytics by type
|
||||
- Supplier spending reports
|
||||
- Monthly summaries
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-12-29
|
||||
**Architecture:** Ultrathin Monolith (integrated module)
|
||||
382
docs/data-entry/README.md
Normal file
382
docs/data-entry/README.md
Normal file
@@ -0,0 +1,382 @@
|
||||
# Data Entry App - Bonuri Fiscale
|
||||
|
||||
Aplicatie pentru introducere bonuri fiscale cu workflow de aprobare si extragere automata date prin OCR.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- Node.js 18+
|
||||
- (Optional) SSH tunnel pentru Oracle nomenclatoare
|
||||
|
||||
### Using Start Script (Recommended)
|
||||
|
||||
```bash
|
||||
# Start all services
|
||||
./start-data-entry.sh
|
||||
|
||||
# Or individual commands:
|
||||
./start-data-entry.sh start # Start all
|
||||
./start-data-entry.sh stop # Stop all
|
||||
./start-data-entry.sh status # Check status
|
||||
./start-data-entry.sh restart backend # Restart backend only
|
||||
```
|
||||
|
||||
**Services:**
|
||||
- Backend: http://localhost:8003
|
||||
- Frontend: http://localhost:3010
|
||||
- API Docs: http://localhost:8003/docs
|
||||
|
||||
### Manual Setup
|
||||
|
||||
#### Backend Setup
|
||||
|
||||
```bash
|
||||
cd data-entry-app/backend
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate # Linux/Mac
|
||||
# sau: venv\Scripts\activate # Windows
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Create .env file
|
||||
cp .env.example .env
|
||||
# Edit .env with your settings
|
||||
|
||||
# Run migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Start server
|
||||
uvicorn app.main:app --reload --port 8003
|
||||
```
|
||||
|
||||
#### Frontend Setup
|
||||
|
||||
```bash
|
||||
cd data-entry-app/frontend
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Start dev server
|
||||
npm run dev -- --port 3010
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
### Pentru Utilizatori
|
||||
- **OCR Automat** - Extragere automata date din poza bonului (suma, data, furnizor, CUI)
|
||||
- Upload poze bonuri fiscale
|
||||
- Completare date bon (suma, data, furnizor)
|
||||
- Selectie tip cheltuiala
|
||||
- Trimitere spre aprobare
|
||||
|
||||
### Pentru Contabili
|
||||
- Vizualizare bonuri in asteptare
|
||||
- Editare note contabile propuse
|
||||
- Aprobare/Respingere bonuri
|
||||
- Aprobare in masa
|
||||
|
||||
## OCR Feature
|
||||
|
||||
### Cum functioneaza
|
||||
|
||||
1. **Upload imagine** - Trage sau selecteaza poza bonului
|
||||
2. **Procesare OCR** - Click pe "Proceseaza cu OCR"
|
||||
3. **Previzualizare** - Datele extrase sunt afisate cu indicatori de incredere
|
||||
4. **Aplicare** - Click "Aplica datele in formular" pentru auto-fill
|
||||
|
||||
### Campuri extrase automat
|
||||
|
||||
| Camp | Acuratete estimata |
|
||||
|------|-------------------|
|
||||
| Suma (TOTAL) | 90-95% |
|
||||
| Data | 85-90% |
|
||||
| Numar bon | 80-85% |
|
||||
| Furnizor | 70-80% |
|
||||
| CUI | 85-90% |
|
||||
| Tip document | 95%+ |
|
||||
|
||||
### OCR System Dependencies (Linux/Docker)
|
||||
|
||||
Pentru functionarea OCR trebuie instalate:
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
apt-get install -y \
|
||||
tesseract-ocr \
|
||||
tesseract-ocr-ron \
|
||||
tesseract-ocr-eng \
|
||||
poppler-utils \
|
||||
libgl1-mesa-glx \
|
||||
libglib2.0-0
|
||||
|
||||
# Fedora/RHEL
|
||||
dnf install -y \
|
||||
tesseract \
|
||||
tesseract-langpack-ron \
|
||||
tesseract-langpack-eng \
|
||||
poppler-utils
|
||||
```
|
||||
|
||||
**Note:** PaddleOCR (engine principal) se instaleaza automat cu pip. Tesseract este folosit ca fallback.
|
||||
|
||||
### OCR System Dependencies (Windows)
|
||||
|
||||
Pe Windows Server trebuie instalate manual urmatoarele componente:
|
||||
|
||||
#### 1. Poppler (pentru conversie PDF → imagini)
|
||||
|
||||
```powershell
|
||||
# Descarca Poppler pentru Windows
|
||||
# https://github.com/osborn/poppler-windows/releases
|
||||
# sau https://github.com/bblanchon/pdfium-binaries
|
||||
|
||||
# Extrage in C:\Program Files\poppler\
|
||||
# Adauga la PATH: C:\Program Files\poppler\Library\bin
|
||||
```
|
||||
|
||||
#### 2. Tesseract OCR (engine OCR backup)
|
||||
|
||||
```powershell
|
||||
# Descarca installer de la:
|
||||
# https://github.com/UB-Mannheim/tesseract/wiki
|
||||
|
||||
# Instaleaza cu limbile: English + Romanian
|
||||
# Default path: C:\Program Files\Tesseract-OCR\
|
||||
# Adauga la PATH
|
||||
```
|
||||
|
||||
#### 3. Python OCR Dependencies (in venv)
|
||||
|
||||
```powershell
|
||||
cd C:\inetpub\wwwroot\roa2web\data-entry-backend
|
||||
.\venv\Scripts\activate
|
||||
|
||||
# Instaleaza dependentele OCR
|
||||
pip install paddlepaddle>=2.5.0
|
||||
pip install paddleocr>=2.7.0
|
||||
pip install opencv-python>=4.8.0
|
||||
pip install pytesseract>=0.3.10
|
||||
pip install pdf2image>=1.16.0
|
||||
|
||||
# Sau din requirements.txt
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
#### 4. Restart serviciu
|
||||
|
||||
```powershell
|
||||
nssm restart ROA2WEB-DataEntry
|
||||
```
|
||||
|
||||
**Note importante Windows:**
|
||||
- Prima rulare PaddleOCR descarca modele (~200MB) - poate dura cateva minute
|
||||
- PaddleOCR necesita ~2GB RAM disponibil
|
||||
- Verifica PATH-ul pentru Poppler si Tesseract dupa instalare
|
||||
- Restart serviciul backend dupa orice modificare PATH
|
||||
|
||||
### OCR API Endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | /api/ocr/status | Check OCR service status |
|
||||
| POST | /api/ocr/extract | Extract data from uploaded image |
|
||||
| POST | /api/ocr/extract-attachment/{id} | Re-process existing attachment |
|
||||
|
||||
### Test OCR
|
||||
|
||||
```bash
|
||||
# Check OCR status
|
||||
curl http://localhost:8003/api/ocr/status
|
||||
|
||||
# Extract from image
|
||||
curl -X POST -F "file=@bon.jpg" http://localhost:8003/api/ocr/extract
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
```
|
||||
DRAFT → PENDING_REVIEW → APPROVED/REJECTED → (SYNCED in Oracle)
|
||||
```
|
||||
|
||||
1. **DRAFT**: Utilizator completeaza datele (manual sau via OCR)
|
||||
2. **PENDING_REVIEW**: Sistemul genereaza note contabile automat
|
||||
3. **APPROVED**: Contabil a aprobat bonul
|
||||
4. **REJECTED**: Contabil a respins (utilizatorul poate corecta)
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
data-entry-app/
|
||||
├── backend/
|
||||
│ ├── app/
|
||||
│ │ ├── main.py # FastAPI entry point
|
||||
│ │ ├── config.py # Settings
|
||||
│ │ ├── db/
|
||||
│ │ │ ├── database.py # SQLite engine
|
||||
│ │ │ ├── models/ # SQLModel models
|
||||
│ │ │ └── crud/ # CRUD operations
|
||||
│ │ ├── schemas/ # Pydantic schemas
|
||||
│ │ │ └── ocr.py # OCR response schemas
|
||||
│ │ ├── services/
|
||||
│ │ │ ├── receipt_service.py
|
||||
│ │ │ ├── ocr_service.py # OCR orchestration
|
||||
│ │ │ ├── ocr_engine.py # PaddleOCR/Tesseract
|
||||
│ │ │ ├── ocr_extractor.py # Regex patterns RO
|
||||
│ │ │ └── image_preprocessor.py # OpenCV pipeline
|
||||
│ │ └── routers/
|
||||
│ │ ├── receipts.py
|
||||
│ │ └── ocr.py # OCR endpoints
|
||||
│ ├── migrations/ # Alembic migrations
|
||||
│ ├── data/
|
||||
│ │ ├── receipts.db # SQLite database
|
||||
│ │ └── uploads/ # Uploaded files
|
||||
│ └── requirements.txt
|
||||
│
|
||||
├── frontend/
|
||||
│ ├── src/
|
||||
│ │ ├── views/receipts/ # Page components
|
||||
│ │ ├── components/
|
||||
│ │ │ ├── receipts/ # Receipt components
|
||||
│ │ │ └── ocr/ # OCR components
|
||||
│ │ │ ├── OCRUploadZone.vue
|
||||
│ │ │ ├── OCRPreview.vue
|
||||
│ │ │ └── OCRConfidenceIndicator.vue
|
||||
│ │ ├── stores/ # Pinia stores
|
||||
│ │ └── router/ # Vue Router
|
||||
│ ├── package.json
|
||||
│ └── vite.config.js
|
||||
│
|
||||
└── docs/ # Documentation
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Backend (.env)
|
||||
|
||||
```bash
|
||||
# SQLite
|
||||
SQLITE_DATABASE_PATH=data/receipts.db
|
||||
|
||||
# File uploads
|
||||
UPLOAD_PATH=data/uploads
|
||||
MAX_UPLOAD_SIZE_MB=10
|
||||
|
||||
# Oracle (for nomenclatures)
|
||||
ORACLE_USER=CONTAFIN_ORACLE
|
||||
ORACLE_PASSWORD=your_password
|
||||
ORACLE_HOST=localhost
|
||||
ORACLE_PORT=1526
|
||||
ORACLE_SID=ROA
|
||||
|
||||
# JWT (shared with reports-app)
|
||||
JWT_SECRET_KEY=your_secret_key
|
||||
JWT_ALGORITHM=HS256
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Create new migration
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
alembic revision --autogenerate -m "Add new field"
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
### Run tests
|
||||
|
||||
```bash
|
||||
# Backend
|
||||
cd backend && pytest
|
||||
|
||||
# Frontend
|
||||
cd frontend && npm run test
|
||||
```
|
||||
|
||||
## API Documentation
|
||||
|
||||
Full API documentation available at http://localhost:8003/docs when backend is running.
|
||||
|
||||
### Key Endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| POST | /api/receipts/ | Create receipt |
|
||||
| GET | /api/receipts/ | List receipts |
|
||||
| GET | /api/receipts/{id} | Get receipt details |
|
||||
| POST | /api/receipts/{id}/submit | Submit for review |
|
||||
| POST | /api/receipts/{id}/approve | Approve receipt |
|
||||
| POST | /api/receipts/{id}/reject | Reject receipt |
|
||||
| POST | /api/receipts/{id}/attachments | Upload attachment |
|
||||
| GET | /api/ocr/status | OCR service status |
|
||||
| POST | /api/ocr/extract | OCR image extraction |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### OCR not working
|
||||
|
||||
1. Check OCR status: `curl http://localhost:8003/api/ocr/status`
|
||||
2. Install system dependencies (tesseract, poppler)
|
||||
3. Verify PaddleOCR installed: `python -c "from paddleocr import PaddleOCR"`
|
||||
|
||||
### OCR Windows - "poppler not in PATH"
|
||||
|
||||
```powershell
|
||||
# Eroare: "Unable to get page count. Is poppler installed and in PATH?"
|
||||
|
||||
# Solutie 1: Adauga Poppler la PATH
|
||||
# System Properties → Environment Variables → System variables → Path → New
|
||||
# Adauga: C:\Program Files\poppler\Library\bin
|
||||
|
||||
# Solutie 2: Restart serviciul dupa modificarea PATH
|
||||
nssm restart ROA2WEB-DataEntry
|
||||
|
||||
# Verificare:
|
||||
pdfinfo --version
|
||||
```
|
||||
|
||||
### OCR Windows - "tesseract not found"
|
||||
|
||||
```powershell
|
||||
# Eroare: "tesseract is not installed or it's not in your PATH"
|
||||
|
||||
# Solutie: Adauga Tesseract la PATH
|
||||
# C:\Program Files\Tesseract-OCR\
|
||||
|
||||
# Verificare:
|
||||
tesseract --version
|
||||
tesseract --list-langs # Trebuie sa arate 'ron' si 'eng'
|
||||
```
|
||||
|
||||
### OCR Windows - PaddleOCR import error
|
||||
|
||||
```powershell
|
||||
# Eroare: "No module named 'paddleocr'"
|
||||
|
||||
cd C:\inetpub\wwwroot\roa2web\data-entry-backend
|
||||
.\venv\Scripts\activate
|
||||
pip install paddlepaddle>=2.5.0
|
||||
pip install paddleocr>=2.7.0
|
||||
|
||||
# Restart serviciu
|
||||
nssm restart ROA2WEB-DataEntry
|
||||
```
|
||||
|
||||
### Low OCR accuracy
|
||||
|
||||
- Ensure good lighting when taking receipt photos
|
||||
- Keep receipt flat (no folds/wrinkles)
|
||||
- Try PDF instead of JPG for scanned documents
|
||||
- Check if text is in focus
|
||||
|
||||
## Phase 2 (Future)
|
||||
|
||||
- Oracle sync for approved receipts
|
||||
- Integration with pack_contafin procedures
|
||||
- Automatic posting to ACT/RUL tables
|
||||
429
docs/telegram/README.md
Normal file
429
docs/telegram/README.md
Normal file
@@ -0,0 +1,429 @@
|
||||
# ROA2WEB Telegram Bot
|
||||
|
||||
> **Telegram Frontend for ROA2WEB ERP System** with Direct Command Interface
|
||||
|
||||
## Overview
|
||||
|
||||
ROA2WEB Telegram Bot provides a command-based interface to the ROA2WEB Financial ERP system through Telegram. Users can access dashboards, invoices, treasury data, and company information using simple slash commands.
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Direct Command Interface**: Simple `/` commands for all operations
|
||||
- **Simplified Authentication** 🆕: Multiple linking methods (Deep Link, QR Code, Manual) - 77% faster than before!
|
||||
- **One-Click Connection**: Deep link automatically opens Telegram with pre-populated code
|
||||
- **QR Code Support**: Scan from mobile to link instantly (cross-device)
|
||||
- **Secure Authentication**: Account linking with Oracle backend and JWT token management
|
||||
- **Financial Data Access**: Query dashboards, invoices, treasury data, and company information
|
||||
- **Company Selection**: Set active company for all subsequent queries
|
||||
- **Multi-language**: Romanian and English support
|
||||
- **Session Management**: Active company persistence with SQLite database
|
||||
- **Docker Ready**: Containerized deployment with Docker Compose
|
||||
|
||||
## Architecture
|
||||
|
||||
### System Components
|
||||
|
||||
```
|
||||
telegram-bot/
|
||||
├── app/
|
||||
│ ├── agent/ # Session management
|
||||
│ │ └── session.py # Active company persistence
|
||||
│ ├── api/ # Backend API client
|
||||
│ │ └── client.py # HTTP client for ROA2WEB backend
|
||||
│ ├── auth/ # Authentication & linking
|
||||
│ │ └── linking.py # Account linking logic
|
||||
│ ├── bot/ # Telegram bot handlers
|
||||
│ │ ├── handlers.py # Command handlers
|
||||
│ │ ├── helpers.py # Helper functions
|
||||
│ │ ├── formatters.py # Response formatting
|
||||
│ │ └── keyboards.py # Inline keyboard helpers
|
||||
│ ├── db/ # SQLite database (standalone)
|
||||
│ │ ├── database.py # Connection & schema
|
||||
│ │ └── operations.py # CRUD operations
|
||||
│ ├── internal_api.py # FastAPI for backend communication
|
||||
│ └── main.py # Main entry point
|
||||
├── data/ # SQLite database storage (gitignored)
|
||||
├── tests/ # Unit & integration tests
|
||||
├── Dockerfile # Container configuration
|
||||
├── requirements.txt # Python dependencies
|
||||
└── .env.example # Environment variables template
|
||||
```
|
||||
|
||||
### Data Flow
|
||||
|
||||
1. **User sends command** → Telegram → Bot Command Handlers
|
||||
2. **Authentication check** → SQLite database → JWT validation
|
||||
3. **API calls** → ROA2WEB Backend → Oracle database
|
||||
4. **Response formatting** → Telegram user
|
||||
|
||||
### Database Schema (SQLite)
|
||||
|
||||
The bot uses a standalone SQLite database for:
|
||||
|
||||
- **telegram_users**: User accounts and Oracle account linking
|
||||
- **telegram_auth_codes**: Temporary 8-character linking codes (15 min expiry)
|
||||
- **telegram_sessions**: Active company selection and session state
|
||||
|
||||
## Installation & Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.11+
|
||||
- Telegram account
|
||||
- ROA2WEB backend API running (default: http://localhost:8001)
|
||||
|
||||
### Step 1: Create Telegram Bot
|
||||
|
||||
1. Open Telegram and search for `@BotFather`
|
||||
2. Send `/newbot` command
|
||||
3. Follow prompts to create bot
|
||||
4. Save the bot token provided
|
||||
|
||||
### Step 2: Install Dependencies
|
||||
|
||||
```bash
|
||||
# Navigate to telegram-bot directory
|
||||
cd reports-app/telegram-bot
|
||||
|
||||
# Create virtual environment
|
||||
python3 -m venv venv
|
||||
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate # Linux/Mac
|
||||
# or
|
||||
venv\Scripts\activate # Windows
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Step 3: Configure Environment
|
||||
|
||||
```bash
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env file with your configuration
|
||||
nano .env
|
||||
```
|
||||
|
||||
Required configuration in `.env`:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
TELEGRAM_BOT_TOKEN=your_bot_token_from_botfather
|
||||
BACKEND_URL=http://localhost:8001
|
||||
|
||||
# Database
|
||||
SQLITE_DB_PATH=./data/telegram_bot.db
|
||||
|
||||
# Internal API (for backend communication)
|
||||
INTERNAL_API_PORT=8002
|
||||
|
||||
# Optional
|
||||
LOG_LEVEL=INFO
|
||||
SENTRY_DSN=https://your-sentry-dsn
|
||||
ENVIRONMENT=production
|
||||
```
|
||||
|
||||
### Step 4: Run the Bot
|
||||
|
||||
```bash
|
||||
# Make sure backend is running first
|
||||
# Backend should be at http://localhost:8001 (or configured BACKEND_URL)
|
||||
|
||||
# Run the bot
|
||||
python -m app.main
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Available Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `/start [code]` | Start bot or link account with 8-char code |
|
||||
| `/help` | Show available commands and usage guide |
|
||||
| `/companies` | View accessible companies/firms |
|
||||
| `/selectcompany [name]` | Select or search for active company |
|
||||
| `/dashboard` | View financial dashboard for active company |
|
||||
| `/sold` | View balance (alias for `/dashboard`) |
|
||||
| `/facturi [filter]` | View invoices (optional filters: neplatite, platite) |
|
||||
| `/trezorerie` | View treasury/payment data |
|
||||
| `/clear` | Clear active company selection |
|
||||
| `/unlink` | Unlink Telegram account from Oracle |
|
||||
|
||||
### Authentication Flow (Updated 2025 - FAZA 1 Improvements)
|
||||
|
||||
The bot now supports **3 easy methods** for account linking:
|
||||
|
||||
#### Method 1: Deep Link (Recommended - One Click) 🚀
|
||||
|
||||
1. **Login** to ROA2WEB web application at http://localhost:3000
|
||||
2. **Navigate** to `/telegram` (Settings → Telegram)
|
||||
3. **Click** "Generează Cod" button
|
||||
4. **Click** "🚀 Deschide în Telegram" button
|
||||
5. **Telegram opens automatically** with code pre-populated
|
||||
6. **Done!** Account linked in <30 seconds
|
||||
|
||||
**Benefits:**
|
||||
- ⚡ **Zero manual copying** - code is pre-filled
|
||||
- 🎯 **One click** - Telegram opens automatically
|
||||
- ⏱️ **Super fast** - complete in ~30 seconds
|
||||
- 📱 **Works on desktop and mobile** (same device)
|
||||
|
||||
#### Method 2: QR Code (Cross-Device) 📷
|
||||
|
||||
Perfect when working on desktop but have Telegram on mobile:
|
||||
|
||||
1. **Login** to ROA2WEB web app (desktop)
|
||||
2. **Navigate** to `/telegram`
|
||||
3. **Click** "Generează Cod"
|
||||
4. **Scan QR Code** displayed on screen with Telegram on mobile
|
||||
5. **Telegram opens** with code pre-populated
|
||||
6. **Done!** Account linked
|
||||
|
||||
**Benefits:**
|
||||
- 📱 **Cross-device** - desktop browser → mobile Telegram
|
||||
- 📷 **Easy scanning** - just point camera at screen
|
||||
- ✅ **No typing** - code automatically loaded
|
||||
|
||||
#### Method 3: Manual (Traditional Fallback) ⌨️
|
||||
|
||||
If deep link or QR code don't work:
|
||||
|
||||
1. **Login** to ROA2WEB web application
|
||||
2. **Navigate** to `/telegram`
|
||||
3. **Click** "Generează Cod"
|
||||
4. **Click** copy button (📋) to copy code
|
||||
5. **Open Telegram** manually
|
||||
6. **Send code** to bot:
|
||||
- **Direct input**: `ABC123XY` (just paste)
|
||||
- **Classic format**: `/start ABC123XY`
|
||||
7. **Done!** Account linked
|
||||
|
||||
**Technical Details:**
|
||||
- Backend generates **8-character code** (valid 15 minutes)
|
||||
- Backend saves code to telegram-bot via internal API (`POST /internal/save-code`)
|
||||
- Bot verifies code and links accounts in SQLite database
|
||||
- Bot receives **JWT token** from backend
|
||||
- User can now use commands to query financial data
|
||||
|
||||
**Improvement Metrics:**
|
||||
- ⏱️ Time: **3 minutes → 40 seconds** (77% reduction)
|
||||
- 📉 Steps: **7 → 4** (43% reduction)
|
||||
- ✅ Manual copying: **Eliminated** (with Method 1 or 2)
|
||||
|
||||
### Usage Examples
|
||||
|
||||
```
|
||||
User: /companies
|
||||
Bot: Lists all accessible companies with ID, name, and CUI
|
||||
|
||||
User: /selectcompany ACME
|
||||
Bot: Shows selection keyboard for companies matching "ACME"
|
||||
[User clicks company button]
|
||||
Bot: Company selected: ACME SRL
|
||||
|
||||
User: /dashboard
|
||||
Bot: Displays dashboard statistics for ACME SRL:
|
||||
- Total balance
|
||||
- Invoices issued/paid/unpaid
|
||||
- Total collections and payments
|
||||
|
||||
User: /facturi neplatite
|
||||
Bot: Shows list of unpaid invoices for ACME SRL
|
||||
|
||||
User: /trezorerie
|
||||
Bot: Displays treasury data (cash balance, bank accounts, etc.)
|
||||
|
||||
User: /clear
|
||||
Bot: Active company cleared. Use /selectcompany to select another.
|
||||
```
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
### Build Image
|
||||
|
||||
```bash
|
||||
# From telegram-bot directory
|
||||
docker build -t roa-telegram-bot .
|
||||
```
|
||||
|
||||
### Run Container
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name roa-telegram-bot \
|
||||
-e TELEGRAM_BOT_TOKEN=your_token \
|
||||
-e BACKEND_URL=http://backend:8001 \
|
||||
-v $(pwd)/data:/app/data \
|
||||
roa-telegram-bot
|
||||
```
|
||||
|
||||
### Docker Compose
|
||||
|
||||
See `docker-compose.yml` in project root for full orchestration with backend and database.
|
||||
|
||||
## Development
|
||||
|
||||
### Project Structure
|
||||
|
||||
- **app/main.py**: Bot entry point and Telegram application setup
|
||||
- **app/bot/handlers.py**: All command handlers (`/start`, `/dashboard`, etc.)
|
||||
- **app/bot/helpers.py**: Helper functions for company selection and prompts
|
||||
- **app/bot/formatters.py**: Response formatting utilities
|
||||
- **app/auth/linking.py**: Account linking and JWT management
|
||||
- **app/api/client.py**: HTTP client for backend API calls
|
||||
- **app/agent/session.py**: Session management (active company persistence)
|
||||
- **app/db/**: SQLite database operations
|
||||
- **app/internal_api.py**: FastAPI for backend callbacks
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest tests/ -v
|
||||
|
||||
# Run specific test suites
|
||||
pytest tests/test_auth.py -v # Authentication tests
|
||||
pytest tests/test_session_company.py -v # Session & company tests
|
||||
pytest tests/test_helpers.py -v # Helper function tests
|
||||
pytest tests/test_formatters.py -v # Formatter tests
|
||||
|
||||
# Run with coverage
|
||||
pytest tests/ --cov=app --cov-report=html
|
||||
```
|
||||
|
||||
### Adding New Commands
|
||||
|
||||
1. Add handler function in `app/bot/handlers.py`:
|
||||
```python
|
||||
async def my_command(update: Update, context: ContextTypes.DEFAULT_TYPE):
|
||||
# Implementation
|
||||
pass
|
||||
```
|
||||
|
||||
2. Register handler in `app/main.py`:
|
||||
```python
|
||||
application.add_handler(CommandHandler("mycommand", my_command))
|
||||
```
|
||||
|
||||
3. Add to `__all__` export in `handlers.py`
|
||||
|
||||
## User Experience Improvements (2025)
|
||||
|
||||
### FAZA 1: Authentication Simplification ✅
|
||||
|
||||
**Implemented:** January 2025
|
||||
**Status:** Production Ready
|
||||
|
||||
**What Changed:**
|
||||
- Added **Deep Link** button - one-click Telegram opening
|
||||
- Added **QR Code** generation for cross-device linking
|
||||
- Improved **Manual method** with copy button
|
||||
- Reduced linking time by **77%** (3 min → 40 sec)
|
||||
- Reduced steps by **43%** (7 → 4 steps)
|
||||
|
||||
**Documentation:**
|
||||
- **Complete Plan:** [TELEGRAM_AUTH_IMPROVEMENT_PLAN.md](./TELEGRAM_AUTH_IMPROVEMENT_PLAN.md)
|
||||
- **Testing Guide:** [TESTING_INSTRUCTIONS_FAZA1.md](./TESTING_INSTRUCTIONS_FAZA1.md)
|
||||
- **Implementation Summary:** [FAZA1_IMPLEMENTATION_SUMMARY.md](./FAZA1_IMPLEMENTATION_SUMMARY.md)
|
||||
|
||||
**Frontend Changes:**
|
||||
- `reports-app/frontend/src/views/TelegramView.vue` - Complete UI refactor
|
||||
- Added `qrcode.vue` dependency for QR generation
|
||||
- Environment variable: `VITE_TELEGRAM_BOT_USERNAME`
|
||||
|
||||
### FAZA 2: Email Magic Link (Optional - Future)
|
||||
|
||||
**Status:** Planned
|
||||
**Estimated:** 3.5 hours development
|
||||
|
||||
**What's Planned:**
|
||||
- Email with magic link option
|
||||
- Professional HTML email template
|
||||
- Auto-detect SMTP configuration
|
||||
- Checkbox "Send code via email"
|
||||
|
||||
**Prerequisites:**
|
||||
- SMTP server configuration (Gmail, SendGrid, AWS SES)
|
||||
- User email addresses in Oracle database
|
||||
|
||||
**Note:** FAZA 2 is completely optional. FAZA 1 provides excellent UX for most users.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Bot Not Responding
|
||||
|
||||
- Check bot is running: `ps aux | grep python.*main.py`
|
||||
- Check logs: `tail -f logs/telegram-bot.log`
|
||||
- Verify TELEGRAM_BOT_TOKEN is correct
|
||||
- Ensure backend is accessible at BACKEND_URL
|
||||
|
||||
### Authentication Issues
|
||||
|
||||
- Check auth code expiry (15 minutes)
|
||||
- Verify backend internal API is accessible (port 8002)
|
||||
- Check SQLite database permissions
|
||||
- **Deep Link not working?** Browser may block protocol handler - click "Allow" when prompted
|
||||
- **QR Code not scanning?** Ensure good lighting and camera focus
|
||||
- **Manual method always works** as fallback
|
||||
|
||||
### Database Issues
|
||||
|
||||
- Ensure `data/` directory exists and is writable
|
||||
- Check database file: `sqlite3 data/telegram_bot.db ".schema"`
|
||||
- Automatic cleanup runs hourly for expired data
|
||||
|
||||
### Deep Link Issues (FAZA 1)
|
||||
|
||||
**Problem:** Deep link button doesn't open Telegram
|
||||
|
||||
**Solutions:**
|
||||
1. **Desktop:** Browser may ask permission - click "Allow" to open Telegram
|
||||
2. **Mobile:** Ensure Telegram app is installed
|
||||
3. **Fallback:** Use QR Code (Method 2) or Manual (Method 3)
|
||||
|
||||
**Browser Compatibility:**
|
||||
- ✅ Chrome/Edge - Works perfectly
|
||||
- ⚠️ Firefox - May show permission prompt
|
||||
- ⚠️ Safari - May show permission prompt
|
||||
- ✅ Mobile browsers - Works on all major browsers
|
||||
|
||||
**Problem:** QR Code doesn't display
|
||||
|
||||
**Solutions:**
|
||||
1. Check browser console for errors (F12)
|
||||
2. Ensure `qrcode.vue` package is installed: `npm list qrcode.vue`
|
||||
3. Rebuild frontend: `cd frontend && npm run build`
|
||||
4. **Fallback:** Use Manual method (always works)
|
||||
|
||||
## API Integration
|
||||
|
||||
The bot communicates with ROA2WEB backend via HTTP API:
|
||||
|
||||
- `POST /api/telegram/auth/verify-user` - Verify user_id
|
||||
- `POST /api/telegram/auth/refresh-token` - Refresh JWT
|
||||
- `GET /api/companies` - Get user companies
|
||||
- `GET /api/dashboard/{company_id}` - Dashboard data
|
||||
- `GET /api/invoices/{company_id}` - Invoice list
|
||||
- `GET /api/treasury/{company_id}` - Treasury data
|
||||
|
||||
All requests include JWT token in Authorization header (except verify-user endpoint).
|
||||
|
||||
## Security
|
||||
|
||||
- JWT tokens are stored encrypted in SQLite
|
||||
- Auth codes expire after 15 minutes
|
||||
- Sessions are automatically cleaned up
|
||||
- Environment variables never committed to git
|
||||
- Rate limiting on authentication endpoints
|
||||
|
||||
## License
|
||||
|
||||
See main project LICENSE file.
|
||||
|
||||
## Support
|
||||
|
||||
For issues, questions, or contributions, see the main ROA2WEB project documentation.
|
||||
1928
docs/telegram/TELEGRAM_BUTTON_INTERFACE_PLAN.md
Normal file
1928
docs/telegram/TELEGRAM_BUTTON_INTERFACE_PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
496
docs/telegram/testing/DOCKER_TESTING_GUIDE.md
Normal file
496
docs/telegram/testing/DOCKER_TESTING_GUIDE.md
Normal file
@@ -0,0 +1,496 @@
|
||||
# 🐳 Docker Testing Guide - ROA2WEB Telegram Bot
|
||||
|
||||
This guide provides instructions for testing the Telegram bot Docker deployment.
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
Before testing Docker deployment:
|
||||
|
||||
- [ ] Docker Engine installed (version 20.10+)
|
||||
- [ ] Docker Compose installed (version 2.0+)
|
||||
- [ ] At least 2GB RAM available for containers
|
||||
- [ ] At least 10GB disk space available
|
||||
|
||||
## 🔍 Pre-Build Verification
|
||||
|
||||
### 1. Check Dockerfile Syntax
|
||||
|
||||
```bash
|
||||
cd /path/to/roa2web/reports-app/telegram-bot
|
||||
|
||||
# Verify Dockerfile exists and is valid
|
||||
cat Dockerfile
|
||||
|
||||
# Check for common issues
|
||||
grep -n "COPY\|RUN\|ENV" Dockerfile
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- Multi-stage build with `builder` and `production` stages
|
||||
- Non-root user `telegrambot` created
|
||||
- All required dependencies installed
|
||||
- Tini init system configured
|
||||
- Health check defined
|
||||
|
||||
### 2. Verify Required Files
|
||||
|
||||
```bash
|
||||
# Check all required files exist
|
||||
ls -la app/
|
||||
ls -la requirements.txt
|
||||
ls -la .env.example
|
||||
ls -la Dockerfile
|
||||
```
|
||||
|
||||
**Required files**:
|
||||
- ✅ `Dockerfile`
|
||||
- ✅ `requirements.txt`
|
||||
- ✅ `.env.example`
|
||||
- ✅ `app/` directory with all modules
|
||||
- ✅ `.dockerignore` (optional but recommended)
|
||||
|
||||
### 3. Check .dockerignore
|
||||
|
||||
```bash
|
||||
cat .dockerignore
|
||||
```
|
||||
|
||||
**Should exclude**:
|
||||
- `venv/`
|
||||
- `__pycache__/`
|
||||
- `*.pyc`
|
||||
- `.env`
|
||||
- `data/*.db`
|
||||
- `.git/`
|
||||
- `tests/`
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Build Tests
|
||||
|
||||
### Test 1: Build Telegram Bot Image
|
||||
|
||||
```bash
|
||||
cd /path/to/roa2web/reports-app/telegram-bot
|
||||
|
||||
# Build the image
|
||||
docker build -t roa2web/telegram-bot:test --target production .
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Build completes without errors
|
||||
- ✅ Both stages (builder + production) execute
|
||||
- ✅ Final image size < 500MB (typically ~300-400MB)
|
||||
- ✅ No security warnings in output
|
||||
|
||||
**Troubleshooting**:
|
||||
- If build fails at requirements install → check `requirements.txt` syntax
|
||||
- If permission errors → ensure Dockerfile uses correct user
|
||||
- If large image size → verify multi-stage build is working
|
||||
|
||||
### Test 2: Inspect Built Image
|
||||
|
||||
```bash
|
||||
# Check image size
|
||||
docker images roa2web/telegram-bot:test
|
||||
|
||||
# Inspect image details
|
||||
docker inspect roa2web/telegram-bot:test
|
||||
|
||||
# Check layers
|
||||
docker history roa2web/telegram-bot:test
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- Image size: 300-450 MB
|
||||
- Base: `python:3.11-slim`
|
||||
- User: `telegrambot` (not root)
|
||||
- Working dir: `/app`
|
||||
- Health check configured
|
||||
|
||||
### Test 3: Build with Docker Compose
|
||||
|
||||
```bash
|
||||
cd /path/to/roa2web
|
||||
|
||||
# Build telegram-bot service
|
||||
docker-compose build roa-telegram-bot
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Service builds successfully
|
||||
- ✅ Image tagged as `roa2web/telegram-bot:latest`
|
||||
- ✅ No errors or warnings
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Runtime Tests
|
||||
|
||||
### Test 4: Run Standalone Container (Without Backend)
|
||||
|
||||
```bash
|
||||
# Create test .env file
|
||||
cat > .env.test <<EOF
|
||||
TELEGRAM_BOT_TOKEN=test_token_here
|
||||
CLAUDE_API_KEY=test_api_key_here
|
||||
BACKEND_URL=http://localhost:8000
|
||||
SQLITE_DB_PATH=/app/data/telegram_bot.db
|
||||
INTERNAL_API_PORT=8002
|
||||
LOG_LEVEL=DEBUG
|
||||
EOF
|
||||
|
||||
# Run container in test mode
|
||||
docker run --rm \
|
||||
--name telegram-bot-test \
|
||||
--env-file .env.test \
|
||||
-p 8002:8002 \
|
||||
-v $(pwd)/data:/app/data \
|
||||
roa2web/telegram-bot:test
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Container starts without errors
|
||||
- ✅ Logs show "Telegram bot starting..."
|
||||
- ✅ Database initialized at `/app/data/telegram_bot.db`
|
||||
- ✅ Internal API listening on port 8002
|
||||
- ⚠️ May fail to connect to Telegram API (expected without valid token)
|
||||
|
||||
**Verify**:
|
||||
```bash
|
||||
# In another terminal:
|
||||
# Check container is running
|
||||
docker ps | grep telegram-bot-test
|
||||
|
||||
# Check logs
|
||||
docker logs telegram-bot-test
|
||||
|
||||
# Test internal API health endpoint
|
||||
curl http://localhost:8002/internal/health
|
||||
```
|
||||
|
||||
### Test 5: Run with Docker Compose (Full Stack)
|
||||
|
||||
```bash
|
||||
cd /path/to/roa2web
|
||||
|
||||
# Ensure .env file exists with all required variables
|
||||
cp .env.example .env
|
||||
# Edit .env and add:
|
||||
# - TELEGRAM_BOT_TOKEN
|
||||
# - CLAUDE_API_KEY
|
||||
# - ORACLE credentials
|
||||
# - JWT secret
|
||||
|
||||
# Start just the telegram-bot service (depends on backend)
|
||||
docker-compose up roa-telegram-bot
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Dependencies start first: `roa-redis`, `roa-ssh-tunnel`, `roa-backend`
|
||||
- ✅ Backend health check passes
|
||||
- ✅ Telegram bot starts and connects to backend
|
||||
- ✅ SQLite database persists in `telegram-bot-data` volume
|
||||
|
||||
**Troubleshooting**:
|
||||
```bash
|
||||
# Check service status
|
||||
docker-compose ps
|
||||
|
||||
# View logs
|
||||
docker-compose logs roa-telegram-bot
|
||||
docker-compose logs roa-backend
|
||||
|
||||
# Check networks
|
||||
docker network ls | grep roa-network
|
||||
docker network inspect roa-network
|
||||
```
|
||||
|
||||
### Test 6: Health Check
|
||||
|
||||
```bash
|
||||
# Wait for service to be healthy
|
||||
docker-compose ps roa-telegram-bot
|
||||
|
||||
# Should show: (healthy)
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Health check passes after ~40 seconds (start_period)
|
||||
- ✅ Health check endpoint returns 200 OK
|
||||
- ✅ Service status shows `(healthy)`
|
||||
|
||||
**Manual health check**:
|
||||
```bash
|
||||
# Access container
|
||||
docker exec -it roa-telegram-bot /bin/bash
|
||||
|
||||
# Inside container:
|
||||
python -c "import httpx; import asyncio; asyncio.run(httpx.AsyncClient().get('http://localhost:8002/internal/health'))"
|
||||
|
||||
# Should output: 200 OK
|
||||
```
|
||||
|
||||
### Test 7: Database Persistence
|
||||
|
||||
```bash
|
||||
# Start service
|
||||
docker-compose up -d roa-telegram-bot
|
||||
|
||||
# Check database file exists
|
||||
docker exec roa-telegram-bot ls -la /app/data/
|
||||
|
||||
# Expected: telegram_bot.db file
|
||||
|
||||
# Stop and remove container
|
||||
docker-compose down
|
||||
|
||||
# Start again
|
||||
docker-compose up -d roa-telegram-bot
|
||||
|
||||
# Verify data persisted
|
||||
docker exec roa-telegram-bot sqlite3 /app/data/telegram_bot.db "SELECT COUNT(*) FROM telegram_users;"
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Database file persists across container restarts
|
||||
- ✅ Data remains intact in `telegram-bot-data` volume
|
||||
- ✅ No data loss
|
||||
|
||||
### Test 8: Environment Variables
|
||||
|
||||
```bash
|
||||
# Check environment variables are set
|
||||
docker exec roa-telegram-bot env | grep -E "TELEGRAM|CLAUDE|BACKEND|SQLITE"
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
```
|
||||
TELEGRAM_BOT_TOKEN=<your_token>
|
||||
CLAUDE_API_KEY=<your_key>
|
||||
BACKEND_URL=http://roa-backend:8000
|
||||
SQLITE_DB_PATH=/app/data/telegram_bot.db
|
||||
INTERNAL_API_PORT=8002
|
||||
```
|
||||
|
||||
### Test 9: Network Connectivity
|
||||
|
||||
```bash
|
||||
# Test bot can reach backend
|
||||
docker exec roa-telegram-bot curl -v http://roa-backend:8000/health
|
||||
|
||||
# Test backend can reach bot internal API
|
||||
docker exec roa-backend curl -v http://roa-telegram-bot:8002/internal/health
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Both services can communicate via `roa-network`
|
||||
- ✅ DNS resolution works (service names resolve)
|
||||
- ✅ Health endpoints return 200 OK
|
||||
|
||||
### Test 10: Logs and Monitoring
|
||||
|
||||
```bash
|
||||
# View real-time logs
|
||||
docker-compose logs -f roa-telegram-bot
|
||||
|
||||
# View last 100 lines
|
||||
docker-compose logs --tail=100 roa-telegram-bot
|
||||
|
||||
# Search for errors
|
||||
docker-compose logs roa-telegram-bot | grep -i error
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Logs are readable and structured
|
||||
- ✅ No critical errors
|
||||
- ✅ Log level respects `LOG_LEVEL` env var
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Tests
|
||||
|
||||
### Test 11: User Permissions
|
||||
|
||||
```bash
|
||||
# Check container is not running as root
|
||||
docker exec roa-telegram-bot whoami
|
||||
# Expected: telegrambot
|
||||
|
||||
# Check file permissions
|
||||
docker exec roa-telegram-bot ls -la /app/
|
||||
# Expected: All files owned by telegrambot:telegrambot
|
||||
```
|
||||
|
||||
### Test 12: Port Exposure
|
||||
|
||||
```bash
|
||||
# Check exposed ports
|
||||
docker port roa-telegram-bot
|
||||
|
||||
# Should only show:
|
||||
# 8002/tcp -> 0.0.0.0:8002
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Only internal API port (8002) exposed
|
||||
- ✅ No unnecessary ports open
|
||||
|
||||
### Test 13: Volume Mounts
|
||||
|
||||
```bash
|
||||
# Check volumes
|
||||
docker volume inspect roa2web_telegram-bot-data
|
||||
|
||||
# Check mount point
|
||||
docker inspect roa-telegram-bot | grep -A 10 Mounts
|
||||
```
|
||||
|
||||
**Expected**:
|
||||
- ✅ Only `/app/data` is mounted
|
||||
- ✅ Volume is named `telegram-bot-data`
|
||||
- ✅ No sensitive files mounted
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Integration Tests
|
||||
|
||||
### Test 14: Full Stack Integration
|
||||
|
||||
```bash
|
||||
# Start all services
|
||||
cd /path/to/roa2web
|
||||
docker-compose up -d
|
||||
|
||||
# Wait for all services to be healthy
|
||||
docker-compose ps
|
||||
|
||||
# Test complete flow:
|
||||
# 1. Backend generates auth code
|
||||
# 2. Bot verifies code
|
||||
# 3. User links account
|
||||
# 4. Bot queries backend API
|
||||
```
|
||||
|
||||
**Test Steps**:
|
||||
|
||||
1. **Generate Auth Code via Backend**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/telegram/auth/generate-code \
|
||||
-H "Authorization: Bearer <jwt_token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"telegram_user_id": 123456}'
|
||||
```
|
||||
|
||||
2. **Verify Code in Bot Database**:
|
||||
```bash
|
||||
docker exec roa-telegram-bot sqlite3 /app/data/telegram_bot.db \
|
||||
"SELECT * FROM telegram_auth_codes WHERE telegram_user_id = 123456;"
|
||||
```
|
||||
|
||||
3. **Link via Telegram Bot**:
|
||||
- Send code to bot via Telegram app
|
||||
- Verify linking succeeds
|
||||
|
||||
4. **Query Dashboard**:
|
||||
- Ask bot: "Show dashboard for company 1"
|
||||
- Verify data is retrieved from backend
|
||||
|
||||
---
|
||||
|
||||
## 🛑 Cleanup
|
||||
|
||||
### Remove Test Containers
|
||||
|
||||
```bash
|
||||
# Stop and remove containers
|
||||
docker-compose down
|
||||
|
||||
# Remove volumes (WARNING: deletes data)
|
||||
docker-compose down -v
|
||||
|
||||
# Remove images
|
||||
docker rmi roa2web/telegram-bot:test
|
||||
docker rmi roa2web/telegram-bot:latest
|
||||
```
|
||||
|
||||
### Clean Build Cache
|
||||
|
||||
```bash
|
||||
# Remove build cache
|
||||
docker builder prune -a
|
||||
|
||||
# Remove unused images
|
||||
docker image prune -a
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Test Results Checklist
|
||||
|
||||
| Test ID | Description | Status | Notes |
|
||||
|---------|-------------|--------|-------|
|
||||
| 1 | Build telegram-bot image | ⬜ | |
|
||||
| 2 | Inspect image | ⬜ | |
|
||||
| 3 | Build with docker-compose | ⬜ | |
|
||||
| 4 | Run standalone container | ⬜ | |
|
||||
| 5 | Run with docker-compose | ⬜ | |
|
||||
| 6 | Health check | ⬜ | |
|
||||
| 7 | Database persistence | ⬜ | |
|
||||
| 8 | Environment variables | ⬜ | |
|
||||
| 9 | Network connectivity | ⬜ | |
|
||||
| 10 | Logs and monitoring | ⬜ | |
|
||||
| 11 | User permissions | ⬜ | |
|
||||
| 12 | Port exposure | ⬜ | |
|
||||
| 13 | Volume mounts | ⬜ | |
|
||||
| 14 | Full stack integration | ⬜ | |
|
||||
|
||||
**Overall Result**: ⬜ PASS ⬜ FAIL
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Common Issues & Solutions
|
||||
|
||||
### Issue 1: Build Fails - "requirements.txt not found"
|
||||
**Solution**: Ensure you're in the correct directory (`telegram-bot/`) when building
|
||||
|
||||
### Issue 2: Permission Denied Errors
|
||||
**Solution**: Check Dockerfile uses correct user and permissions are set with `chown`
|
||||
|
||||
### Issue 3: Health Check Fails
|
||||
**Solution**:
|
||||
- Check internal API is starting on port 8002
|
||||
- Verify httpx is installed in requirements.txt
|
||||
- Check logs: `docker logs roa-telegram-bot`
|
||||
|
||||
### Issue 4: Can't Connect to Backend
|
||||
**Solution**:
|
||||
- Ensure both containers are on `roa-network`
|
||||
- Check backend is healthy before starting bot
|
||||
- Use service name `roa-backend` not `localhost`
|
||||
|
||||
### Issue 5: Database Not Persisting
|
||||
**Solution**:
|
||||
- Verify volume is mounted: `docker inspect roa-telegram-bot`
|
||||
- Check volume exists: `docker volume ls | grep telegram-bot-data`
|
||||
- Ensure `/app/data` has write permissions
|
||||
|
||||
---
|
||||
|
||||
## ✅ Success Criteria
|
||||
|
||||
For Docker deployment to be considered successful:
|
||||
|
||||
- ✅ Image builds without errors
|
||||
- ✅ Container starts and runs stably
|
||||
- ✅ Health checks pass
|
||||
- ✅ Bot connects to Telegram API
|
||||
- ✅ Bot connects to backend API
|
||||
- ✅ Database persists across restarts
|
||||
- ✅ No security warnings or vulnerabilities
|
||||
- ✅ Logs are clean (no critical errors)
|
||||
- ✅ All network connectivity works
|
||||
- ✅ Full stack integration succeeds
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-21
|
||||
365
docs/telegram/testing/MANUAL_TESTING_CHECKLIST.md
Normal file
365
docs/telegram/testing/MANUAL_TESTING_CHECKLIST.md
Normal file
@@ -0,0 +1,365 @@
|
||||
# 📋 Manual Testing Checklist - ROA2WEB Telegram Bot
|
||||
|
||||
This checklist guides you through manual testing of the Telegram bot functionality.
|
||||
|
||||
## 🔧 Prerequisites
|
||||
|
||||
Before starting manual tests:
|
||||
|
||||
- [ ] Backend API is running (`http://localhost:8001`)
|
||||
- [ ] SSH tunnel to Oracle DB is active
|
||||
- [ ] Telegram bot is running (`python -m app.main`)
|
||||
- [ ] TELEGRAM_BOT_TOKEN is configured in `.env`
|
||||
- [ ] CLAUDE_API_KEY is configured in `.env` (if using real Claude SDK)
|
||||
- [ ] SQLite database is initialized (`data/telegram_bot.db` exists)
|
||||
|
||||
## 📱 Test Environment Setup
|
||||
|
||||
### Start Services
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start backend API (from roa2web/)
|
||||
cd reports-app/backend
|
||||
source venv/bin/activate
|
||||
uvicorn app.main:app --reload --port 8001
|
||||
|
||||
# Terminal 2: Start Telegram bot
|
||||
cd reports-app/telegram-bot
|
||||
source venv/bin/activate
|
||||
python -m app.main
|
||||
```
|
||||
|
||||
### Test User Setup
|
||||
|
||||
- [ ] Create test Oracle user account in Oracle database (if needed)
|
||||
- [ ] Have test Telegram account ready (@testuser or similar)
|
||||
- [ ] Know the Telegram user ID (can be found via bot command `/start`)
|
||||
|
||||
---
|
||||
|
||||
## ✅ Test Cases
|
||||
|
||||
### 1. Bot Discovery & Initial Contact
|
||||
|
||||
**Test 1.1: Start Bot**
|
||||
- [ ] Open Telegram and search for `@ROA2WEBBot`
|
||||
- [ ] Click "Start" or send `/start` command
|
||||
- [ ] **Expected**: Bot responds with welcome message explaining linking process
|
||||
- [ ] **Expected**: Bot asks for authentication code
|
||||
|
||||
**Test 1.2: Help Command**
|
||||
- [ ] Send `/help` command
|
||||
- [ ] **Expected**: Bot shows list of available commands with descriptions
|
||||
- [ ] **Expected**: Includes `/start`, `/help`, `/clear`, `/companies`, `/unlink`
|
||||
|
||||
---
|
||||
|
||||
### 2. Authentication Flow
|
||||
|
||||
**Test 2.1: Generate Linking Code (via Web)**
|
||||
- [ ] Open web frontend (Vue.js app)
|
||||
- [ ] Login with Oracle credentials
|
||||
- [ ] Navigate to Telegram linking page (if available)
|
||||
- [ ] Click "Generate Telegram Linking Code"
|
||||
- [ ] **Expected**: 8-character code is displayed (e.g., `ABC23456`)
|
||||
- [ ] **Expected**: Code expires in 5 minutes message shown
|
||||
|
||||
**Test 2.2: Link Account with Valid Code**
|
||||
- [ ] In Telegram bot, send the 8-character code from Step 2.1
|
||||
- [ ] **Expected**: Bot responds with "Successfully linked to Oracle account [username]"
|
||||
- [ ] **Expected**: Bot shows list of companies you have access to
|
||||
- [ ] **Expected**: User is now authenticated and can use bot features
|
||||
|
||||
**Test 2.3: Try to Link with Invalid Code**
|
||||
- [ ] Send an invalid code like `INVALID1`
|
||||
- [ ] **Expected**: Bot responds with "Invalid or expired code" message
|
||||
- [ ] **Expected**: Bot prompts to generate new code via web
|
||||
|
||||
**Test 2.4: Try to Link with Expired Code**
|
||||
- [ ] Generate a code via web
|
||||
- [ ] Wait 6+ minutes (past expiration)
|
||||
- [ ] Send expired code to bot
|
||||
- [ ] **Expected**: Bot responds with "Code has expired" message
|
||||
- [ ] **Expected**: Bot suggests generating new code
|
||||
|
||||
**Test 2.5: Try to Reuse Code**
|
||||
- [ ] Generate new code and link successfully
|
||||
- [ ] Unlink account (`/unlink`)
|
||||
- [ ] Try to use the same code again
|
||||
- [ ] **Expected**: Bot rejects code with "Code already used" message
|
||||
|
||||
---
|
||||
|
||||
### 3. User Commands (When Linked)
|
||||
|
||||
**Test 3.1: Companies Command**
|
||||
- [ ] Send `/companies` command
|
||||
- [ ] **Expected**: Bot lists all companies user has access to
|
||||
- [ ] **Expected**: Shows company ID, name, and CUI
|
||||
- [ ] **Expected**: Format is clear and readable
|
||||
|
||||
**Test 3.2: Clear History Command**
|
||||
- [ ] Have some conversation history with bot
|
||||
- [ ] Send `/clear` command
|
||||
- [ ] **Expected**: Bot confirms conversation history cleared
|
||||
- [ ] **Expected**: Bot resets context for new conversation
|
||||
|
||||
**Test 3.3: Unlink Command**
|
||||
- [ ] Send `/unlink` command
|
||||
- [ ] **Expected**: Bot shows confirmation warning
|
||||
- [ ] **Expected**: Shows inline keyboard with "Yes" / "No" buttons
|
||||
- [ ] Press "No" button
|
||||
- [ ] **Expected**: Unlinking cancelled, account still linked
|
||||
- [ ] Send `/unlink` again and press "Yes"
|
||||
- [ ] **Expected**: Account unlinked successfully
|
||||
- [ ] **Expected**: Bot requires new authentication code to continue
|
||||
|
||||
---
|
||||
|
||||
### 4. Conversational Queries (Claude Agent)
|
||||
|
||||
**Note**: These tests require Claude Agent SDK integration to be complete.
|
||||
|
||||
**Test 4.1: Simple Dashboard Query**
|
||||
- [ ] Send message: "Show me the dashboard for company 1"
|
||||
- [ ] **Expected**: Bot retrieves dashboard data
|
||||
- [ ] **Expected**: Shows total balance, invoices count, payments, etc.
|
||||
- [ ] **Expected**: Data is formatted in Romanian language
|
||||
|
||||
**Test 4.2: Invoice Search Query**
|
||||
- [ ] Send: "Find unpaid invoices from October 2025"
|
||||
- [ ] **Expected**: Bot searches invoices with filters
|
||||
- [ ] **Expected**: Returns list of matching invoices
|
||||
- [ ] **Expected**: Shows invoice number, date, client, amount, status
|
||||
|
||||
**Test 4.3: Treasury Query**
|
||||
- [ ] Send: "What's the current treasury status for company 1?"
|
||||
- [ ] **Expected**: Bot retrieves treasury data
|
||||
- [ ] **Expected**: Shows cash balance, bank accounts, payments
|
||||
|
||||
**Test 4.4: Export Request**
|
||||
- [ ] Send: "Export unpaid invoices to Excel"
|
||||
- [ ] **Expected**: Bot generates Excel file
|
||||
- [ ] **Expected**: Sends file via Telegram
|
||||
- [ ] **Expected**: File name includes report type and timestamp
|
||||
- [ ] **Expected**: File can be downloaded and opened
|
||||
|
||||
**Test 4.5: Complex Multi-Step Query**
|
||||
- [ ] Send: "Show me the dashboard, then find invoices over 5000 RON, and export them to PDF"
|
||||
- [ ] **Expected**: Bot handles multi-step request correctly
|
||||
- [ ] **Expected**: Executes each tool in sequence
|
||||
- [ ] **Expected**: Provides updates on progress
|
||||
- [ ] **Expected**: Final PDF file is sent
|
||||
|
||||
**Test 4.6: Romanian Language Support**
|
||||
- [ ] Send messages in Romanian
|
||||
- [ ] **Expected**: Bot understands and responds in Romanian
|
||||
- [ ] **Expected**: Romanian characters displayed correctly (ă, â, î, ș, ț)
|
||||
|
||||
---
|
||||
|
||||
### 5. Error Handling
|
||||
|
||||
**Test 5.1: Query Before Authentication**
|
||||
- [ ] Start new bot conversation (or use fresh account)
|
||||
- [ ] Send query without linking: "Show dashboard"
|
||||
- [ ] **Expected**: Bot responds with "Please authenticate first"
|
||||
- [ ] **Expected**: Bot provides instructions to link account
|
||||
|
||||
**Test 5.2: Invalid Company ID**
|
||||
- [ ] Send: "Show dashboard for company 9999"
|
||||
- [ ] **Expected**: Bot responds with "Company not found" or "No access" message
|
||||
- [ ] **Expected**: Suggests using `/companies` to see available companies
|
||||
|
||||
**Test 5.3: Backend API Offline**
|
||||
- [ ] Stop backend API server
|
||||
- [ ] Try to send query to bot
|
||||
- [ ] **Expected**: Bot responds with "Service temporarily unavailable" message
|
||||
- [ ] **Expected**: Suggests trying again later
|
||||
|
||||
**Test 5.4: Token Expiration**
|
||||
- [ ] Link account and wait for JWT token to expire (30 minutes)
|
||||
- [ ] Send query after expiration
|
||||
- [ ] **Expected**: Bot automatically refreshes token
|
||||
- [ ] **Expected**: Query succeeds without re-authentication
|
||||
|
||||
**Test 5.5: Invalid Export Format**
|
||||
- [ ] Send: "Export dashboard to invalidformat"
|
||||
- [ ] **Expected**: Bot responds with supported formats (xlsx, csv, pdf)
|
||||
- [ ] **Expected**: Asks user to specify valid format
|
||||
|
||||
---
|
||||
|
||||
### 6. Session Management
|
||||
|
||||
**Test 6.1: Conversation Context**
|
||||
- [ ] Send: "Show dashboard for company 1"
|
||||
- [ ] Bot responds with data
|
||||
- [ ] Send follow-up: "Now show invoices"
|
||||
- [ ] **Expected**: Bot remembers company 1 from context
|
||||
- [ ] **Expected**: Shows invoices for company 1
|
||||
|
||||
**Test 6.2: Session Persistence**
|
||||
- [ ] Have conversation with bot
|
||||
- [ ] Stop and restart Telegram bot application
|
||||
- [ ] Resume conversation
|
||||
- [ ] **Expected**: User is still linked (SQLite data persists)
|
||||
- [ ] **Expected**: Can immediately send queries without re-authentication
|
||||
|
||||
**Test 6.3: Multiple Users**
|
||||
- [ ] Use two different Telegram accounts
|
||||
- [ ] Link both to different Oracle users
|
||||
- [ ] Send queries from both accounts simultaneously
|
||||
- [ ] **Expected**: Each user gets their own data
|
||||
- [ ] **Expected**: No data mixing between users
|
||||
- [ ] **Expected**: Sessions isolated correctly
|
||||
|
||||
---
|
||||
|
||||
### 7. Database Operations
|
||||
|
||||
**Test 7.1: Check User Record**
|
||||
```bash
|
||||
# In terminal
|
||||
sqlite3 data/telegram_bot.db
|
||||
SELECT * FROM telegram_users;
|
||||
```
|
||||
- [ ] **Expected**: User record exists with telegram_user_id
|
||||
- [ ] **Expected**: oracle_username is populated after linking
|
||||
- [ ] **Expected**: jwt_token and token_expires_at are set
|
||||
|
||||
**Test 7.2: Check Auth Codes**
|
||||
```sql
|
||||
SELECT * FROM telegram_auth_codes WHERE oracle_username = 'testuser';
|
||||
```
|
||||
- [ ] **Expected**: Used codes have `used_at` timestamp
|
||||
- [ ] **Expected**: Expired codes have `expires_at` in the past
|
||||
|
||||
**Test 7.3: Database Cleanup**
|
||||
- [ ] Generate expired auth code (wait 6 minutes or manually update DB)
|
||||
- [ ] Wait for cleanup task to run (runs hourly)
|
||||
- [ ] **Expected**: Expired codes are removed from database
|
||||
- [ ] **Expected**: Database size doesn't grow indefinitely
|
||||
|
||||
---
|
||||
|
||||
### 8. Performance & Reliability
|
||||
|
||||
**Test 8.1: Response Time**
|
||||
- [ ] Send simple query: "Show dashboard"
|
||||
- [ ] Measure time from send to receive response
|
||||
- [ ] **Expected**: Response within 3-5 seconds
|
||||
- [ ] **Expected**: No timeouts
|
||||
|
||||
**Test 8.2: Large Data Export**
|
||||
- [ ] Request export of large dataset (100+ invoices)
|
||||
- [ ] **Expected**: Bot handles large exports gracefully
|
||||
- [ ] **Expected**: File generates successfully
|
||||
- [ ] **Expected**: File size is reasonable (<10MB for typical data)
|
||||
|
||||
**Test 8.3: Concurrent Requests**
|
||||
- [ ] Send multiple queries rapidly (3-4 in quick succession)
|
||||
- [ ] **Expected**: All queries are processed
|
||||
- [ ] **Expected**: Responses arrive in correct order
|
||||
- [ ] **Expected**: No crashes or errors
|
||||
|
||||
---
|
||||
|
||||
### 9. Security Tests
|
||||
|
||||
**Test 9.1: Unauthorized Access**
|
||||
- [ ] Without linking, try to call backend API directly with fake token
|
||||
- [ ] **Expected**: Backend rejects request with 401 Unauthorized
|
||||
|
||||
**Test 9.2: Token in Database**
|
||||
```bash
|
||||
sqlite3 data/telegram_bot.db
|
||||
SELECT jwt_token FROM telegram_users LIMIT 1;
|
||||
```
|
||||
- [ ] **Expected**: Token exists in database
|
||||
- [ ] **Note**: Ensure database file is properly secured in production
|
||||
- [ ] **Note**: Database should not be committed to git
|
||||
|
||||
**Test 9.3: Code Security**
|
||||
- [ ] Generate linking code
|
||||
- [ ] Try to guess codes by brute force
|
||||
- [ ] **Expected**: Codes are random and hard to guess (8 chars, no ambiguous chars)
|
||||
- [ ] **Expected**: Codes expire after 5 minutes
|
||||
|
||||
---
|
||||
|
||||
## 📊 Test Results
|
||||
|
||||
### Summary
|
||||
|
||||
| Test Category | Total Tests | Passed | Failed | Skipped |
|
||||
|--------------|-------------|--------|--------|---------|
|
||||
| Bot Discovery | 2 | - | - | - |
|
||||
| Authentication Flow | 5 | - | - | - |
|
||||
| User Commands | 3 | - | - | - |
|
||||
| Conversational Queries | 6 | - | - | - |
|
||||
| Error Handling | 5 | - | - | - |
|
||||
| Session Management | 3 | - | - | - |
|
||||
| Database Operations | 3 | - | - | - |
|
||||
| Performance | 3 | - | - | - |
|
||||
| Security | 3 | - | - | - |
|
||||
| **TOTAL** | **33** | **0** | **0** | **0** |
|
||||
|
||||
### Failed Tests
|
||||
|
||||
_List any failed tests here with details:_
|
||||
|
||||
| Test ID | Description | Error | Notes |
|
||||
|---------|-------------|-------|-------|
|
||||
| - | - | - | - |
|
||||
|
||||
### Notes & Issues
|
||||
|
||||
_Document any issues discovered during testing:_
|
||||
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Reporting Issues
|
||||
|
||||
If you find bugs during manual testing:
|
||||
|
||||
1. **Document**:
|
||||
- Test case ID
|
||||
- Steps to reproduce
|
||||
- Expected behavior
|
||||
- Actual behavior
|
||||
- Error messages (if any)
|
||||
- Screenshots (if applicable)
|
||||
|
||||
2. **Check Database State**:
|
||||
```bash
|
||||
sqlite3 data/telegram_bot.db
|
||||
# Inspect relevant tables
|
||||
```
|
||||
|
||||
3. **Check Logs**:
|
||||
- Telegram bot logs (console output)
|
||||
- Backend API logs
|
||||
- SQLite database queries
|
||||
|
||||
4. **Create Issue**:
|
||||
- File bug in project issue tracker
|
||||
- Include all documentation from step 1
|
||||
|
||||
---
|
||||
|
||||
## ✅ Test Completion
|
||||
|
||||
**Tester Name**: _______________
|
||||
|
||||
**Date**: _______________
|
||||
|
||||
**Overall Result**: [ ] PASS [ ] FAIL
|
||||
|
||||
**Sign-off**: _______________
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-21
|
||||
213
docs/telegram/testing/README_INTEGRATION_TESTS.md
Normal file
213
docs/telegram/testing/README_INTEGRATION_TESTS.md
Normal file
@@ -0,0 +1,213 @@
|
||||
# Integration Tests Guide
|
||||
|
||||
This directory contains both **unit tests** (with mocks) and **integration tests** (with real data).
|
||||
|
||||
## Test Categories
|
||||
|
||||
### Unit Tests (Default)
|
||||
- **Files**: `test_*.py` (except `*_real*.py`)
|
||||
- **Dependencies**: None (all mocked)
|
||||
- **Speed**: Fast (~2-3 seconds)
|
||||
- **Run by**: CI/CD, developers
|
||||
- **Command**: `pytest` (runs by default)
|
||||
|
||||
**Examples**:
|
||||
- `test_auth.py` - Authentication flow tests
|
||||
- `test_tools.py` - Claude Agent tools tests
|
||||
- `test_helpers.py` - Bot helper functions tests
|
||||
- `test_formatters.py` - Response formatters tests
|
||||
- `test_session_company.py` - Session management tests
|
||||
|
||||
### Integration Tests (Manual)
|
||||
- **Files**: `test_helpers_real*.py`
|
||||
- **Dependencies**: Backend API + Database/Environment
|
||||
- **Speed**: Slower (~10-30 seconds)
|
||||
- **Run by**: Developers manually
|
||||
- **Command**: `pytest -m integration`
|
||||
- **Marked with**: `@pytest.mark.integration`
|
||||
|
||||
**Examples**:
|
||||
- `test_helpers_real.py` - Integration tests with SQLite DB
|
||||
- `test_helpers_real_simple.py` - Integration tests with direct API auth
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Run All Unit Tests (Default)
|
||||
```bash
|
||||
# Runs all tests EXCEPT integration tests
|
||||
pytest
|
||||
|
||||
# Explicit: run only unit tests
|
||||
pytest -m "not integration"
|
||||
```
|
||||
|
||||
### Run Integration Tests
|
||||
```bash
|
||||
# Run only integration tests
|
||||
pytest -m integration
|
||||
|
||||
# Run specific integration test file
|
||||
pytest tests/test_helpers_real.py -m integration
|
||||
|
||||
# Run as standalone script (alternative)
|
||||
python tests/test_helpers_real_simple.py
|
||||
```
|
||||
|
||||
### Run ALL Tests (Unit + Integration)
|
||||
```bash
|
||||
# Override default filter
|
||||
pytest -m ""
|
||||
```
|
||||
|
||||
## Integration Test Requirements
|
||||
|
||||
### For `test_helpers_real.py`:
|
||||
- ✅ Backend API running on `localhost:8001`
|
||||
- ✅ SQLite database (`data/telegram_bot.db`) with at least one linked user
|
||||
- ⚠️ Requires existing user session in database
|
||||
|
||||
### For `test_helpers_real_simple.py`:
|
||||
- ✅ Backend API running on `localhost:8001`
|
||||
- ✅ Environment variables set:
|
||||
```bash
|
||||
export TEST_USERNAME="your_oracle_username"
|
||||
export TEST_PASSWORD="your_oracle_password"
|
||||
```
|
||||
- ✅ Valid Oracle credentials for backend authentication
|
||||
|
||||
## Setting Up Integration Tests
|
||||
|
||||
### 1. Start Backend API
|
||||
```bash
|
||||
cd roa2web/reports-app/backend
|
||||
source venv/bin/activate
|
||||
uvicorn app.main:app --reload --port 8001
|
||||
```
|
||||
|
||||
### 2. Set Credentials (for `test_helpers_real_simple.py`)
|
||||
```bash
|
||||
# In your shell or .env file
|
||||
export TEST_USERNAME="MARIUS M" # Your Oracle username
|
||||
export TEST_PASSWORD="your_password" # Your Oracle password
|
||||
```
|
||||
|
||||
### 3. Run Integration Tests
|
||||
```bash
|
||||
cd roa2web/reports-app/telegram-bot
|
||||
source venv/bin/activate
|
||||
pytest -m integration -v
|
||||
```
|
||||
|
||||
## CI/CD Configuration
|
||||
|
||||
Integration tests are **automatically skipped** in CI/CD pipelines because:
|
||||
- They require external services (backend API, database)
|
||||
- They need real credentials
|
||||
- Default pytest configuration excludes them: `-m "not integration"`
|
||||
|
||||
To run them in CI/CD, you would need to:
|
||||
1. Set up backend API service
|
||||
2. Provide TEST_USERNAME and TEST_PASSWORD as secrets
|
||||
3. Override pytest command: `pytest -m ""`
|
||||
|
||||
## Markers Reference
|
||||
|
||||
Defined in `pytest.ini`:
|
||||
|
||||
```ini
|
||||
markers =
|
||||
unit: Unit tests with mocks (fast, no external dependencies)
|
||||
integration: Integration tests with real backend/database (slow, requires setup)
|
||||
slow: Slow tests that take more than 1 second
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
pytest -m unit # Run only unit tests
|
||||
pytest -m integration # Run only integration tests
|
||||
pytest -m slow # Run only slow tests
|
||||
pytest -m "not slow" # Skip slow tests
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### When Writing Tests
|
||||
|
||||
**Unit Tests (Preferred for most cases)**:
|
||||
- ✅ Fast and reliable
|
||||
- ✅ No external dependencies
|
||||
- ✅ Use mocks (`unittest.mock`, `AsyncMock`)
|
||||
- ✅ Test one component at a time
|
||||
- ✅ Run in CI/CD
|
||||
|
||||
**Integration Tests (Use sparingly)**:
|
||||
- ⚠️ Slower and can be flaky
|
||||
- ⚠️ Require full environment setup
|
||||
- ⚠️ Test multiple components together
|
||||
- ⚠️ Manual execution only
|
||||
- ✅ Useful for validation before releases
|
||||
- ✅ Document real usage patterns
|
||||
|
||||
### Coverage Goals
|
||||
|
||||
- **Unit tests**: Aim for 80%+ code coverage
|
||||
- **Integration tests**: Focus on critical paths and end-to-end flows
|
||||
- Don't duplicate: If unit tests cover it well, integration tests may be redundant
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Integration Tests Fail
|
||||
1. **Check backend is running**: `curl http://localhost:8001/health`
|
||||
2. **Verify credentials**: Ensure `TEST_USERNAME` and `TEST_PASSWORD` are set
|
||||
3. **Check database**: Ensure `data/telegram_bot.db` exists and has users
|
||||
4. **Review logs**: Check backend logs for API errors
|
||||
|
||||
### Integration Tests Skipped
|
||||
- This is normal! They're skipped by default.
|
||||
- Use `pytest -m integration` to run them explicitly.
|
||||
|
||||
### Import Errors
|
||||
```bash
|
||||
# Make sure you're in the right directory
|
||||
cd /mnt/e/proiecte/roa2web/roa2web/reports-app/telegram-bot
|
||||
|
||||
# Activate virtual environment
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Example Output
|
||||
|
||||
### Unit Tests (Default)
|
||||
```bash
|
||||
$ pytest
|
||||
============================= test session starts ==============================
|
||||
collected 93 items / 5 deselected / 88 selected
|
||||
|
||||
tests/test_auth.py ............ [ 13%]
|
||||
tests/test_formatters.py ................ [ 31%]
|
||||
tests/test_helpers.py .................. [ 51%]
|
||||
tests/test_session_company.py .................. [ 72%]
|
||||
tests/test_tools.py .................. [100%]
|
||||
|
||||
======================== 88 passed, 5 deselected in 3.42s ======================
|
||||
```
|
||||
|
||||
### Integration Tests (Explicit)
|
||||
```bash
|
||||
$ pytest -m integration
|
||||
============================= test session starts ==============================
|
||||
collected 93 items / 88 deselected / 5 selected
|
||||
|
||||
tests/test_helpers_real.py .... [ 80%]
|
||||
tests/test_helpers_real_simple.py . [100%]
|
||||
|
||||
======================== 5 passed, 88 deselected in 12.37s =====================
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-22
|
||||
**Author**: Claude Code Session
|
||||
Reference in New Issue
Block a user