feat: Add JWT auth and nomenclature sync to data-entry-app
Integrate shared JWT authentication into data-entry-app: - Add Oracle pool initialization for auth service - Add AuthenticationMiddleware to protect API routes - Update all receipt endpoints to use CurrentUser from JWT - Add shared auth router (/api/auth/login, /api/auth/refresh) Add nomenclature synchronization feature: - Create SQLite models for synced suppliers, local suppliers, and cash registers - Add nomenclature router with sync triggers and CRUD endpoints - Add sync service for Oracle → SQLite nomenclature data - Update nomenclature_service to use synced SQLite data with fallbacks Create shared frontend components: - Add shared/frontend/ with LoginView.vue, auth store factory, login.css - Integrate shared login and auth into data-entry-app frontend - Add axios-based API service with token refresh interceptor 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
273
data-entry-app/backend/NOMENCLATURE_SYNC.md
Normal file
273
data-entry-app/backend/NOMENCLATURE_SYNC.md
Normal file
@@ -0,0 +1,273 @@
|
||||
# Nomenclature Sync - Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the implementation of FAZA 3: Nomenclature Sync for the Data Entry App.
|
||||
|
||||
The nomenclature sync system allows the data-entry-app to maintain a local SQLite cache of nomenclatures (suppliers, cash registers) from Oracle, reducing the need for live Oracle queries and improving performance.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Database Tables
|
||||
|
||||
Three new SQLite tables were added:
|
||||
|
||||
1. **synced_suppliers** - Suppliers synced from Oracle NOM_PARTENERI
|
||||
- `oracle_id` - Original Oracle ID
|
||||
- `company_id` - Company this supplier belongs to
|
||||
- `name` - Supplier name
|
||||
- `fiscal_code` - CUI/CIF
|
||||
- `address` - Supplier address
|
||||
- `synced_at` - Last sync timestamp
|
||||
|
||||
2. **local_suppliers** - Suppliers created locally from OCR (not in Oracle)
|
||||
- `company_id` - Company ID
|
||||
- `name` - Supplier name
|
||||
- `fiscal_code` - CUI/CIF
|
||||
- `address` - Supplier address
|
||||
- `created_by` - Username who created it
|
||||
- `pending_oracle_sync` - Flag for future Oracle sync
|
||||
|
||||
3. **synced_cash_registers** - Cash registers and bank accounts from Oracle
|
||||
- `oracle_id` - Original Oracle ID
|
||||
- `company_id` - Company ID
|
||||
- `name` - Register name
|
||||
- `account_code` - Account code (5311, 5121, etc.)
|
||||
- `register_type` - 'cash' or 'bank'
|
||||
- `synced_at` - Last sync timestamp
|
||||
|
||||
### Components
|
||||
|
||||
#### 1. Models (`app/db/models/nomenclature.py`)
|
||||
SQLModel models for the three tables above.
|
||||
|
||||
#### 2. Sync Service (`app/services/sync_service.py`)
|
||||
Core business logic for syncing nomenclatures:
|
||||
|
||||
- `sync_suppliers()` - Sync suppliers from Oracle to SQLite
|
||||
- `sync_cash_registers()` - Sync cash registers from Oracle to SQLite
|
||||
- `search_supplier()` - Search in synced + local suppliers
|
||||
- `create_local_supplier()` - Create local supplier from OCR data
|
||||
- `get_all_suppliers()` - Get all suppliers for dropdown
|
||||
- `get_all_cash_registers()` - Get all cash registers for dropdown
|
||||
|
||||
#### 3. API Router (`app/routers/nomenclature.py`)
|
||||
New API endpoints:
|
||||
|
||||
**GET /api/nomenclature/suppliers**
|
||||
- Get all suppliers (synced + local) for dropdown/autocomplete
|
||||
- Query params: `search`, `company_id`
|
||||
- Returns: List of SupplierOption
|
||||
|
||||
**GET /api/nomenclature/suppliers/search**
|
||||
- Search for supplier by fiscal code or name
|
||||
- Query params: `fiscal_code`, `name`, `company_id`
|
||||
- Returns: SupplierSearchResult (found, supplier, source)
|
||||
|
||||
**POST /api/nomenclature/suppliers/local**
|
||||
- Create a local supplier from OCR data
|
||||
- Body: LocalSupplierCreate (name, fiscal_code, address)
|
||||
- Returns: LocalSupplierResponse
|
||||
|
||||
**GET /api/nomenclature/cash-registers**
|
||||
- Get all cash registers for a company
|
||||
- Query params: `company_id`
|
||||
- Returns: List of CashRegisterOption
|
||||
|
||||
**POST /api/nomenclature/sync/suppliers**
|
||||
- Manually trigger supplier sync from Oracle
|
||||
- Returns: SyncResult (synced count, errors)
|
||||
|
||||
**POST /api/nomenclature/sync/cash-registers**
|
||||
- Manually trigger cash register sync from Oracle
|
||||
- Returns: SyncResult (synced count, errors)
|
||||
|
||||
**POST /api/nomenclature/sync/all**
|
||||
- Sync all nomenclatures (suppliers + cash registers)
|
||||
- Returns: Combined sync results
|
||||
|
||||
#### 4. Updated Services
|
||||
|
||||
**nomenclature_service.py** was updated to use synced data:
|
||||
- `get_partners()` - Now accepts optional `session` parameter, returns synced data if available, falls back to mock
|
||||
- `get_cash_registers()` - Now accepts optional `session` parameter, returns synced data if available, falls back to mock
|
||||
|
||||
**receipts.py router** was updated to pass session to nomenclature service.
|
||||
|
||||
## Company Schema Mapping
|
||||
|
||||
The sync service needs to know which Oracle schema to query for each company. This is configured in `sync_service.py`:
|
||||
|
||||
```python
|
||||
COMPANY_SCHEMAS = {
|
||||
1: "CONTAFIN",
|
||||
2: "CONTAFIN2",
|
||||
}
|
||||
```
|
||||
|
||||
**TODO**: Move this to a config table or environment variable for production.
|
||||
|
||||
## Oracle Integration
|
||||
|
||||
The sync service connects to Oracle using the shared `oracle_pool` from `/shared/database/oracle_pool.py`.
|
||||
|
||||
**Prerequisites**:
|
||||
- SSH tunnel must be running (development/Linux)
|
||||
- Oracle connection pool must be initialized
|
||||
- Environment variables must be set (ORACLE_USER, ORACLE_PASSWORD, ORACLE_HOST, ORACLE_PORT, ORACLE_SID)
|
||||
|
||||
**Oracle Tables Used**:
|
||||
- `{schema}.NOM_PARTENERI` - Suppliers (WHERE ACTIV = 1)
|
||||
- `{schema}.NOM_CASE` - Cash registers (WHERE ACTIV = 1)
|
||||
|
||||
**Note**: Table and column names may need adjustment based on actual Oracle schema.
|
||||
|
||||
## Usage Flow
|
||||
|
||||
### Initial Setup (One-time)
|
||||
|
||||
1. Ensure Oracle connection is available:
|
||||
```bash
|
||||
# Start SSH tunnel (if on Linux/dev)
|
||||
./ssh_tunnel.sh start
|
||||
```
|
||||
|
||||
2. Run initial sync:
|
||||
```bash
|
||||
# Via API (authenticated request)
|
||||
POST /api/nomenclature/sync/all
|
||||
```
|
||||
|
||||
Or programmatically:
|
||||
```python
|
||||
from app.services.sync_service import SyncService
|
||||
|
||||
# Sync for company 1
|
||||
synced, errors = await SyncService.sync_suppliers(session, company_id=1)
|
||||
synced, errors = await SyncService.sync_cash_registers(session, company_id=1)
|
||||
```
|
||||
|
||||
### Periodic Sync
|
||||
|
||||
Set up a cron job or scheduled task to sync nomenclatures periodically (e.g., daily):
|
||||
|
||||
```python
|
||||
# Example: Add to app lifespan or background task
|
||||
async def sync_all_companies():
|
||||
"""Sync nomenclatures for all companies."""
|
||||
async with get_db_session() as session:
|
||||
for company_id in [1, 2]: # All company IDs
|
||||
await SyncService.sync_suppliers(session, company_id)
|
||||
await SyncService.sync_cash_registers(session, company_id)
|
||||
```
|
||||
|
||||
### Using Synced Data
|
||||
|
||||
The existing endpoints (`/api/receipts/nomenclature/partners`, `/api/receipts/nomenclature/cash-registers`) now automatically use synced data when available.
|
||||
|
||||
**Frontend** - No changes needed! Existing code continues to work:
|
||||
```javascript
|
||||
// Get suppliers (now from synced data)
|
||||
const response = await api.get('/api/receipts/nomenclature/partners?search=OMV');
|
||||
```
|
||||
|
||||
### Creating Local Suppliers from OCR
|
||||
|
||||
When OCR extracts a supplier not in Oracle:
|
||||
|
||||
```javascript
|
||||
// Create local supplier
|
||||
const response = await api.post('/api/nomenclature/suppliers/local', {
|
||||
name: "New Supplier SRL",
|
||||
fiscal_code: "RO12345678",
|
||||
address: "Str. Example 123"
|
||||
});
|
||||
```
|
||||
|
||||
The local supplier will be:
|
||||
- Available immediately in dropdowns
|
||||
- Flagged for future Oracle sync (`pending_oracle_sync = True`)
|
||||
- Created by current user (`created_by = username`)
|
||||
|
||||
## Migration
|
||||
|
||||
Migration: `20251213_002805_add_nomenclature_tables.py`
|
||||
|
||||
Applied with:
|
||||
```bash
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
To rollback:
|
||||
```bash
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. Test sync endpoint:
|
||||
```bash
|
||||
curl -X POST http://localhost:8003/api/nomenclature/sync/suppliers \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
```
|
||||
|
||||
2. Test search:
|
||||
```bash
|
||||
curl "http://localhost:8003/api/nomenclature/suppliers/search?name=OMV" \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
```
|
||||
|
||||
3. Test get all suppliers:
|
||||
```bash
|
||||
curl "http://localhost:8003/api/nomenclature/suppliers" \
|
||||
-H "Authorization: Bearer YOUR_TOKEN"
|
||||
```
|
||||
|
||||
### Unit Tests
|
||||
|
||||
TODO: Add unit tests in `tests/test_sync_service.py`:
|
||||
- Test supplier sync
|
||||
- Test cash register sync
|
||||
- Test search functionality
|
||||
- Test local supplier creation
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Sync fails with "No schema mapping"
|
||||
- Update `COMPANY_SCHEMAS` in `sync_service.py` with correct company-to-schema mappings
|
||||
|
||||
### Sync fails with Oracle connection error
|
||||
- Verify SSH tunnel is running: `./ssh_tunnel.sh status`
|
||||
- Check Oracle credentials in `.env`
|
||||
- Test Oracle connection: `curl http://localhost:8003/health`
|
||||
|
||||
### Tables not found in Oracle
|
||||
- Verify table names in Oracle (may differ from NOM_PARTENERI, NOM_CASE)
|
||||
- Update SQL queries in `sync_service.py` to match actual schema
|
||||
|
||||
### Duplicate suppliers after sync
|
||||
- The sync uses upsert logic (update if exists, insert if new)
|
||||
- Check `oracle_id` + `company_id` uniqueness in synced_suppliers table
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Scheduled Background Sync** - Add cron job or Celery task for automatic daily sync
|
||||
2. **Sync Status Dashboard** - UI to show last sync time, sync statistics
|
||||
3. **Conflict Resolution** - Handle cases where local supplier matches synced supplier
|
||||
4. **Bidirectional Sync** - Push local suppliers to Oracle when approved
|
||||
5. **Incremental Sync** - Only sync changed records (requires last_modified timestamp in Oracle)
|
||||
6. **Multi-Company Support** - Auto-detect user's companies and sync all
|
||||
7. **Sync Notifications** - Notify users when sync completes or fails
|
||||
8. **Audit Log** - Track all sync operations for compliance
|
||||
|
||||
## Related Files
|
||||
|
||||
- Models: `/app/db/models/nomenclature.py`
|
||||
- Service: `/app/services/sync_service.py`
|
||||
- Router: `/app/routers/nomenclature.py`
|
||||
- Migration: `/migrations/versions/20251213_002805_add_nomenclature_tables.py`
|
||||
- Updated: `/app/services/nomenclature_service.py`
|
||||
- Updated: `/app/routers/receipts.py`
|
||||
- Updated: `/app/main.py`
|
||||
Reference in New Issue
Block a user