roa2web-service-auto/data-entry-app/backend/NOMENCLATURE_SYNC.md

Nomenclature Sync - Implementation Guide

Overview

This document describes the implementation of FAZA 3: Nomenclature Sync for the Data Entry App.

The nomenclature sync system allows the data-entry-app to maintain a local SQLite cache of nomenclatures (suppliers, cash registers) from Oracle, reducing the need for live Oracle queries and improving performance.

Architecture

Database Tables

Three new SQLite tables were added:

1. synced_suppliers - Suppliers synced from Oracle NOM_PARTENERI

   • oracle_id - Original Oracle ID
   • company_id - Company this supplier belongs to
   • name - Supplier name
   • fiscal_code - CUI/CIF
   • address - Supplier address
   • synced_at - Last sync timestamp

2. local_suppliers - Suppliers created locally from OCR (not in Oracle)

   • company_id - Company ID
   • name - Supplier name
   • fiscal_code - CUI/CIF
   • address - Supplier address
   • created_by - Username who created it
   • pending_oracle_sync - Flag for future Oracle sync

3. synced_cash_registers - Cash registers and bank accounts from Oracle

   • oracle_id - Original Oracle ID
   • company_id - Company ID
   • name - Register name
   • account_code - Account code (5311, 5121, etc.)
   • register_type - 'cash' or 'bank'
   • synced_at - Last sync timestamp

Components

1. Models (app/db/models/nomenclature.py)

SQLModel models for the three tables above.

2. Sync Service (app/services/sync_service.py)

Core business logic for syncing nomenclatures:

• sync_suppliers() - Sync suppliers from Oracle to SQLite
• sync_cash_registers() - Sync cash registers from Oracle to SQLite
• search_supplier() - Search in synced + local suppliers
• create_local_supplier() - Create local supplier from OCR data
• get_all_suppliers() - Get all suppliers for dropdown
• get_all_cash_registers() - Get all cash registers for dropdown

3. API Router (app/routers/nomenclature.py)

New API endpoints:

GET /api/nomenclature/suppliers

• Get all suppliers (synced + local) for dropdown/autocomplete
• Query params: search, company_id
• Returns: List of SupplierOption

GET /api/nomenclature/suppliers/search

• Search for supplier by fiscal code or name
• Query params: fiscal_code, name, company_id
• Returns: SupplierSearchResult (found, supplier, source)

POST /api/nomenclature/suppliers/local

• Create a local supplier from OCR data
• Body: LocalSupplierCreate (name, fiscal_code, address)
• Returns: LocalSupplierResponse

GET /api/nomenclature/cash-registers

• Get all cash registers for a company
• Query params: company_id
• Returns: List of CashRegisterOption

POST /api/nomenclature/sync/suppliers

• Manually trigger supplier sync from Oracle
• Returns: SyncResult (synced count, errors)

POST /api/nomenclature/sync/cash-registers

• Manually trigger cash register sync from Oracle
• Returns: SyncResult (synced count, errors)

POST /api/nomenclature/sync/all

• Sync all nomenclatures (suppliers + cash registers)
• Returns: Combined sync results

4. Updated Services

nomenclature_service.py was updated to use synced data:

• get_partners() - Now accepts optional session parameter, returns synced data if available, falls back to mock
• get_cash_registers() - Now accepts optional session parameter, returns synced data if available, falls back to mock

receipts.py router was updated to pass session to nomenclature service.

Company Schema Mapping

The sync service needs to know which Oracle schema to query for each company. This is configured in sync_service.py:

COMPANY_SCHEMAS = {
    1: "CONTAFIN",
    2: "CONTAFIN2",
}

TODO: Move this to a config table or environment variable for production.

Oracle Integration

The sync service connects to Oracle using the shared oracle_pool from /shared/database/oracle_pool.py.

Prerequisites:

• SSH tunnel must be running (development/Linux)
• Oracle connection pool must be initialized
• Environment variables must be set (ORACLE_USER, ORACLE_PASSWORD, ORACLE_HOST, ORACLE_PORT, ORACLE_SID)

Oracle Tables Used:

• {schema}.NOM_PARTENERI - Suppliers (WHERE ACTIV = 1)
• {schema}.NOM_CASE - Cash registers (WHERE ACTIV = 1)

Note: Table and column names may need adjustment based on actual Oracle schema.

Usage Flow

Initial Setup (One-time)

1. Ensure Oracle connection is available:

   # Start SSH tunnel (if on Linux/dev)
   ./ssh_tunnel.sh start
   

2. Run initial sync:

   # Via API (authenticated request)
   POST /api/nomenclature/sync/all
   

   Or programmatically:

   from app.services.sync_service import SyncService
   
   # Sync for company 1
   synced, errors = await SyncService.sync_suppliers(session, company_id=1)
   synced, errors = await SyncService.sync_cash_registers(session, company_id=1)
   

Periodic Sync

Set up a cron job or scheduled task to sync nomenclatures periodically (e.g., daily):

# Example: Add to app lifespan or background task
async def sync_all_companies():
    """Sync nomenclatures for all companies."""
    async with get_db_session() as session:
        for company_id in [1, 2]:  # All company IDs
            await SyncService.sync_suppliers(session, company_id)
            await SyncService.sync_cash_registers(session, company_id)

Using Synced Data

The existing endpoints (/api/receipts/nomenclature/partners, /api/receipts/nomenclature/cash-registers) now automatically use synced data when available.

Frontend - No changes needed! Existing code continues to work:

// Get suppliers (now from synced data)
const response = await api.get('/api/receipts/nomenclature/partners?search=OMV');

Creating Local Suppliers from OCR

When OCR extracts a supplier not in Oracle:

// Create local supplier
const response = await api.post('/api/nomenclature/suppliers/local', {
  name: "New Supplier SRL",
  fiscal_code: "RO12345678",
  address: "Str. Example 123"
});

The local supplier will be:

• Available immediately in dropdowns
• Flagged for future Oracle sync (pending_oracle_sync = True)
• Created by current user (created_by = username)

Migration

Migration: 20251213_002805_add_nomenclature_tables.py

Applied with:

alembic upgrade head

To rollback:

alembic downgrade -1

Testing

Manual Testing

1. Test sync endpoint:

   curl -X POST http://localhost:8003/api/nomenclature/sync/suppliers \
     -H "Authorization: Bearer YOUR_TOKEN"
   

2. Test search:

   curl "http://localhost:8003/api/nomenclature/suppliers/search?name=OMV" \
     -H "Authorization: Bearer YOUR_TOKEN"
   

3. Test get all suppliers:

   curl "http://localhost:8003/api/nomenclature/suppliers" \
     -H "Authorization: Bearer YOUR_TOKEN"
   

Unit Tests

TODO: Add unit tests in tests/test_sync_service.py:

• Test supplier sync
• Test cash register sync
• Test search functionality
• Test local supplier creation

Troubleshooting

Sync fails with "No schema mapping"

• Update COMPANY_SCHEMAS in sync_service.py with correct company-to-schema mappings

Sync fails with Oracle connection error

• Verify SSH tunnel is running: ./ssh_tunnel.sh status
• Check Oracle credentials in .env
• Test Oracle connection: curl http://localhost:8003/health

Tables not found in Oracle

• Verify table names in Oracle (may differ from NOM_PARTENERI, NOM_CASE)
• Update SQL queries in sync_service.py to match actual schema

Duplicate suppliers after sync

• The sync uses upsert logic (update if exists, insert if new)
• Check oracle_id + company_id uniqueness in synced_suppliers table

Future Enhancements

1. Scheduled Background Sync - Add cron job or Celery task for automatic daily sync
2. Sync Status Dashboard - UI to show last sync time, sync statistics
3. Conflict Resolution - Handle cases where local supplier matches synced supplier
4. Bidirectional Sync - Push local suppliers to Oracle when approved
5. Incremental Sync - Only sync changed records (requires last_modified timestamp in Oracle)
6. Multi-Company Support - Auto-detect user's companies and sync all
7. Sync Notifications - Notify users when sync completes or fails
8. Audit Log - Track all sync operations for compliance

Related Files

• Models: /app/db/models/nomenclature.py
• Service: /app/services/sync_service.py
• Router: /app/routers/nomenclature.py
• Migration: /migrations/versions/20251213_002805_add_nomenclature_tables.py
• Updated: /app/services/nomenclature_service.py
• Updated: /app/routers/receipts.py
• Updated: /app/main.py