fix telegram

deploy-package-20260223-151231/backend/modules/data_entry/services/ocr/profiles/README.md

@@ -0,0 +1,258 @@
+# Store Profiles - OCR Extraction
+
+Sistem de profile specifice pentru extracție OCR cu hot-reload.
+
+---
+
+## Quick Start: Adaugă un profil nou
+
+```bash
+# 1. Generează profil din PDF-uri (dry-run pentru preview)
+python scripts/generate_store_profile.py \
+  --name "Magazin Nou SRL" \
+  --cui "12345678" \
+  --receipts "docs/data-entry/MagazinNou*.pdf" \
+  --dry-run
+
+# 2. Generează și salvează
+python scripts/generate_store_profile.py \
+  --name "Magazin Nou SRL" \
+  --cui "12345678" \
+  --receipts "docs/data-entry/MagazinNou*.pdf" \
+  --output backend/modules/data_entry/services/ocr/profiles/magazin_nou.py
+
+# 3. Hot-reload (fără restart server)
+curl -X POST http://localhost:8000/api/data-entry/ocr/profiles/reload
+
+# 4. Verifică
+curl http://localhost:8000/api/data-entry/ocr/profiles
+```
+
+---
+
+## Structura directorului
+
+```
+profiles/
+├── __init__.py        # ProfileRegistry + hot-reload (~390 linii)
+├── base.py            # BaseStoreProfile + pattern-uri generice (~410 linii)
+├── lidl.py            # Multi-rate TVA (A/B)
+├── omv.py             # B2B, date YYYY.MM.DD
+├── socar.py           # B2B, date YYYY.MM.DD
+├── brick.py           # Standard TVA
+├── dedeman.py         # E-factura support
+├── kineterra.py       # Non-VAT payer
+├── gama_ink.py        # Standard TVA (toner/cartușe)
+├── electrobering.py   # Standard TVA (electronice)
+├── pictus_velum.py    # Standard TVA (rechizite)
+├── unlimited_keys.py  # Standard TVA, NUMERAR payment
+├── best_print.py      # Non-VAT payer (neplătitor TVA)
+├── stepout_market.py  # TVA 5% (cărți/librărie)
+└── README.md          # Acest fișier
+```
+
+---
+
+## Profile existente (12 profile)
+
+> **Note**: Pattern-urile TVA sunt **flexibile** și acceptă ORICE cotă (5%, 9%, 11%, 19%, 21%, etc.)
+> pentru a gestiona atât datele istorice cât și schimbările viitoare ale legislației.
+
+| Magazin | CUI | Fișier | Caracteristici |
+|---------|-----|--------|----------------|
+| LIDL DISCOUNT S.R.L. | 22891860 | `lidl.py` | Multi-rate TVA (coduri A, B, C, D) |
+| OMV PETROM MARKETING S.R.L. | 11201891 | `omv.py` | B2B (client CUI), date YYYY.MM.DD |
+| SOCAR PETROLEUM S.A. | 12546600 | `socar.py` | B2B (client CUI), date YYYY.MM.DD |
+| FIVE-HOLDING S.A. (BRICK) | 10562600 | `brick.py` | Standard TVA |
+| DEDEMAN SRL | 2816464 | `dedeman.py` | E-factura support |
+| KINETERRA CONCEPT SRL | 31180432 | `kineterra.py` | Non-VAT payer (returnează `[]`) |
+| GAMA INK SERVICE SRL | 17741882 | `gama_ink.py` | Standard TVA (toner, cartușe) |
+| ELECTROBERING S.R.L. | 2744937 | `electrobering.py` | Standard TVA (electronice) |
+| PICTUS VELUM SRL | 39634534 | `pictus_velum.py` | Standard TVA (rechizite) |
+| UNLIMITED KEYS S.R.L. | 18993187 | `unlimited_keys.py` | Standard TVA, **NUMERAR** plată |
+| BEST PRINT TRADE ACTIV SRL | 45417955 | `best_print.py` | **Non-VAT payer** (neplătitor TVA) |
+| STEPOUT MARKET SRL | 35532655 | `stepout_market.py` | TVA 5% (cărți, librărie) |
+
+---
+
+## API Endpoints
+
+| Endpoint | Metodă | Descriere |
+|----------|--------|-----------|
+| `/api/data-entry/ocr/profiles` | GET | Lista toate profilele |
+| `/api/data-entry/ocr/profiles/{cui}` | GET | Detalii profil (acceptă RO prefix) |
+| `/api/data-entry/ocr/profiles/reload` | POST | Hot-reload toate profilele |
+
+### Exemple API
+
+```bash
+# Lista profile
+curl http://localhost:8000/api/data-entry/ocr/profiles \
+  -H "Authorization: Bearer <token>"
+
+# Detalii profil (cu sau fără RO prefix)
+curl http://localhost:8000/api/data-entry/ocr/profiles/22891860
+curl http://localhost:8000/api/data-entry/ocr/profiles/RO22891860
+
+# Hot-reload după modificări
+curl -X POST http://localhost:8000/api/data-entry/ocr/profiles/reload \
+  -H "Authorization: Bearer <token>"
+
+# Response reload:
+{
+  "success": true,
+  "reloaded_modules": 12,
+  "profiles_count": 12,
+  "registered_cuis": ["22891860", "11201891", "12546600", "10562600", ...],
+  "last_reload": "2026-01-06T22:37:05.000000"
+}
+```
+
+---
+
+## Cum funcționează sistemul
+
+### Flow de extracție
+
+```
+ReceiptExtractor.extract()
+  │
+  ├─► STEP 1: Extrage vendor + CUI
+  │     └─► _extract_vendor(), _extract_cui()
+  │
+  ├─► ProfileRegistry.get_profile(cui)
+  │     └─► Returnează profil specific sau None
+  │
+  ├─► STEP 2: Extracție cu profil (dacă există)
+  │     ├─► profile.extract_total()
+  │     ├─► profile.extract_date()
+  │     ├─► profile.extract_receipt_number()
+  │     ├─► profile.extract_tva_entries()
+  │     ├─► profile.extract_payment_methods()
+  │     └─► profile.extract_client_cui()
+  │
+  └─► STEP 3-4: Validare + post-procesare
+```
+
+### Fallback
+
+Dacă nu există profil pentru CUI, se folosește logica generică din `ReceiptExtractor`.
+
+---
+
+## Structura unui profil
+
+```python
+from .base import BaseStoreProfile
+from . import ProfileRegistry
+
+@ProfileRegistry.register
+class MagazinNouProfile(BaseStoreProfile):
+    """Docstring cu descriere magazin."""
+
+    CUI_LIST = ["12345678"]  # Poate avea mai multe CUI-uri
+    NAME_PATTERNS = ["MAGAZIN", "MAGAZIN NOU", "MAG4ZIN"]  # OCR variants
+    STORE_NAME = "Magazin Nou SRL"
+
+    # Override doar ce e diferit de base class
+    def extract_tva_entries(self, text: str) -> List[dict]:
+        # Pattern-uri specifice magazinului
+        ...
+
+    def get_validation_hints(self) -> Dict[str, Any]:
+        return {
+            "has_multi_rate_tva": False,
+            "card_equals_total": True,
+            "has_client_cui": False,
+            "has_efactura": False,
+            "is_non_vat_payer": False,
+        }
+```
+
+---
+
+## Pattern-uri disponibile în base.py
+
+BaseStoreProfile include pattern-uri generice OCR-tolerant:
+
+| Pattern | Descriere |
+|---------|-----------|
+| `TOTAL_PATTERNS` | 8 variante pentru TOTAL (TOTAL:, TOTAL DE PLATA, etc.) |
+| `DATE_PATTERNS` | 6 variante (DD.MM.YYYY, YYYY-MM-DD, DD/MM/YYYY) |
+| `DATE_PATTERNS_OCR_SPACES` | 4 variante cu spații OCR ("2025. 08. 14") |
+| `NUMBER_PATTERNS` | 11 variante pentru număr bon (NDS, BF, C3POS) |
+| `PAYMENT_PATTERNS` | 8 variante pentru CARD/NUMERAR |
+| `CLIENT_MARKERS` | 6 variante pentru secțiune CLIENT |
+| `CLIENT_CUI_PATTERNS` | 7 variante pentru CUI client |
+
+### Metode implementate în base class
+
+- `extract_total(text)` → `Tuple[Decimal, float]`
+- `extract_date(text)` → `Tuple[date, float]`
+- `extract_receipt_number(text)` → `Tuple[str, float]`
+- `extract_payment_methods(text)` → `List[dict]`
+- `extract_client_cui(text)` → `Tuple[str, float]`
+- `extract_client_name(text)` → `Tuple[str, float]`
+
+---
+
+## Când ai nevoie de profil custom?
+
+| Situație | Exemplu | Ce trebuie override |
+|----------|---------|---------------------|
+| **Multi-rate TVA** | Lidl (TVA A, TVA B) | `extract_tva_entries()` |
+| **Format dată special** | OMV/Socar (YYYY.MM.DD) | `DATE_PATTERNS_OCR_SPACES` |
+| **B2B receipts** | Benzinării (au client CUI) | `extract_client_cui()` |
+| **Non-VAT payer** | Kineterra | `extract_tva_entries()` returnează `[]` |
+| **E-factura** | Dedeman | `extract_efactura_reference()` |
+
+---
+
+## Decizii de design
+
+1. **Hot-reload manual** - endpoint `/profiles/reload` apelat când se modifică fișiere
+2. **Persistență în Python** - profile în Git, version controlled
+3. **Fallback graceful** - dacă nu există profil, folosește logica generică
+4. **CUI normalization** - gestionează automat prefixul "RO" și whitespace
+5. **Deduplicare TVA** - folosește `seen = set()` pentru a evita duplicate
+
+---
+
+## Comenzi utile
+
+```bash
+# Verifică syntax Python pentru toate profilele
+for f in backend/modules/data_entry/services/ocr/profiles/*.py; do
+  python3 -m py_compile "$f" && echo "✓ $(basename $f)"
+done
+
+# Lista profile
+ls -la backend/modules/data_entry/services/ocr/profiles/
+
+# Pornește backend pentru testare
+cd backend && source venv/bin/activate
+uvicorn main:app --host 0.0.0.0 --port 8000 --workers 1
+
+# Test OCR pe un PDF
+curl -X POST -F "file=@docs/data-entry/test.pdf" \
+  -H "Authorization: Bearer <token>" \
+  "http://localhost:8000/api/data-entry/ocr/extract?engine=doctr_plus"
+```
+
+---
+
+## Script generare profile
+
+`scripts/generate_store_profile.py` - generator automat de profile
+
+```bash
+# Vezi help
+python scripts/generate_store_profile.py --help
+
+# Funcționalități:
+# - Analizează PDF-uri via OCR API
+# - Detectează: TVA format, date format, payment patterns, B2B
+# - Generează cod Python cu OCR error variants
+# - Suportă glob patterns (*.pdf)
+# - Verifică sintaxa după generare
+```

deploy-package-20260223-151231/backend/modules/data_entry/services/ocr/profiles/__init__.py

@@ -0,0 +1,398 @@
+"""
+Store Profiles Registry with Hot-Reload Support.
+
+This module provides a registry for store-specific OCR extraction profiles.
+Profiles can be reloaded at runtime without restarting the server.
+
+Usage:
+    from backend.modules.data_entry.services.ocr.profiles import ProfileRegistry
+
+    # Get profile for a CUI
+    profile = ProfileRegistry.get_profile("22891860")
+    if profile:
+        tva_entries = profile.extract_tva_entries(text)
+
+    # Reload all profiles (after file changes)
+    count = ProfileRegistry.reload_all()
+
+Architecture:
+    - ProfileRegistry: Singleton registry with class methods
+    - BaseStoreProfile: Abstract base class for profiles
+    - @ProfileRegistry.register: Decorator for profile classes
+
+Hot-Reload Mechanism:
+    1. Admin calls POST /profiles/reload endpoint
+    2. Registry clears instance cache
+    3. importlib.reload() re-executes each profile module
+    4. @register decorator re-registers classes with new code
+"""
+
+from __future__ import annotations
+
+import importlib
+import logging
+import sys
+from datetime import datetime
+from pathlib import Path
+from typing import Dict, List, Optional, Type, TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from .base import BaseStoreProfile
+
+logger = logging.getLogger(__name__)
+
+# Directory containing profile modules
+PROFILES_DIR = Path(__file__).parent
+
+
+class ProfileRegistry:
+    """
+    Registry for store-specific OCR extraction profiles.
+
+    Uses class methods for singleton-like behavior without explicit instantiation.
+    Supports hot-reload via importlib.reload() for runtime updates.
+
+    Attributes:
+        _profiles: Maps CUI -> profile class (not instance)
+        _instances: Maps CUI -> profile instance (lazy, cleared on reload)
+        _last_reload: Timestamp of last reload
+        _loaded: Whether initial load has been performed
+    """
+
+    # Class-level storage (singleton pattern via class methods)
+    _profiles: Dict[str, Type["BaseStoreProfile"]] = {}
+    _instances: Dict[str, "BaseStoreProfile"] = {}
+    _last_reload: Optional[datetime] = None
+    _loaded: bool = False
+
+    # -------------------------------------------------------------------------
+    # Registration
+    # -------------------------------------------------------------------------
+
+    @classmethod
+    def register(cls, profile_class: Type["BaseStoreProfile"]) -> Type["BaseStoreProfile"]:
+        """
+        Decorator to register a store profile class.
+
+        Registers the profile for all CUIs in the class's CUI_LIST.
+        Safe for re-registration during hot-reload (overwrites existing).
+
+        Usage:
+            @ProfileRegistry.register
+            class LidlProfile(BaseStoreProfile):
+                CUI_LIST = ["22891860"]
+                ...
+
+        Args:
+            profile_class: Profile class to register
+
+        Returns:
+            The same class (allows use as decorator)
+
+        Raises:
+            ValueError: If CUI_LIST is empty
+        """
+        cui_list = getattr(profile_class, 'CUI_LIST', [])
+        store_name = getattr(profile_class, 'STORE_NAME', profile_class.__name__)
+
+        if not cui_list:
+            logger.warning(f"Profile {profile_class.__name__} has empty CUI_LIST, skipping")
+            return profile_class
+
+        # Register for each CUI
+        for cui in cui_list:
+            # Normalize CUI (remove RO prefix, strip whitespace)
+            normalized_cui = cls._normalize_cui(cui)
+
+            if normalized_cui in cls._profiles:
+                old_class = cls._profiles[normalized_cui]
+                logger.debug(
+                    f"Re-registering CUI {normalized_cui}: "
+                    f"{old_class.__name__} -> {profile_class.__name__}"
+                )
+                # Clear cached instance for this CUI
+                cls._instances.pop(normalized_cui, None)
+
+            cls._profiles[normalized_cui] = profile_class
+            logger.debug(f"Registered profile {profile_class.__name__} for CUI {normalized_cui}")
+
+        logger.info(f"Registered {store_name} for CUIs: {cui_list}")
+        return profile_class
+
+    # -------------------------------------------------------------------------
+    # Lookup
+    # -------------------------------------------------------------------------
+
+    @classmethod
+    def get_profile(cls, cui: Optional[str]) -> Optional["BaseStoreProfile"]:
+        """
+        Get profile instance for a CUI.
+
+        Uses lazy instantiation - creates instance on first access.
+        Returns None if no profile is registered for this CUI.
+
+        Args:
+            cui: CUI to lookup (with or without RO prefix)
+
+        Returns:
+            Profile instance or None
+        """
+        if not cui:
+            return None
+
+        # Ensure profiles are loaded
+        if not cls._loaded:
+            cls._load_all_profiles()
+
+        normalized_cui = cls._normalize_cui(cui)
+
+        # Check if profile exists
+        profile_class = cls._profiles.get(normalized_cui)
+        if not profile_class:
+            return None
+
+        # Lazy instantiation
+        if normalized_cui not in cls._instances:
+            try:
+                cls._instances[normalized_cui] = profile_class()
+                logger.debug(f"Instantiated {profile_class.__name__} for CUI {normalized_cui}")
+            except Exception as e:
+                logger.error(f"Failed to instantiate {profile_class.__name__}: {e}")
+                return None
+
+        return cls._instances[normalized_cui]
+
+    @classmethod
+    def has_profile(cls, cui: Optional[str]) -> bool:
+        """Check if a profile exists for this CUI."""
+        if not cui:
+            return False
+        if not cls._loaded:
+            cls._load_all_profiles()
+        return cls._normalize_cui(cui) in cls._profiles
+
+    # -------------------------------------------------------------------------
+    # Listing
+    # -------------------------------------------------------------------------
+
+    @classmethod
+    def list_profiles(cls) -> List[Dict]:
+        """
+        List all registered profiles.
+
+        Returns:
+            List of dicts with cui, class_name, store_name, name_patterns
+        """
+        if not cls._loaded:
+            cls._load_all_profiles()
+
+        result = []
+        seen_classes = set()
+
+        for cui, profile_class in cls._profiles.items():
+            # Avoid duplicates for profiles with multiple CUIs
+            if profile_class.__name__ in seen_classes:
+                continue
+            seen_classes.add(profile_class.__name__)
+
+            result.append({
+                "cuis": list(getattr(profile_class, 'CUI_LIST', [])),
+                "class_name": profile_class.__name__,
+                "store_name": getattr(profile_class, 'STORE_NAME', profile_class.__name__),
+                "name_patterns": list(getattr(profile_class, 'NAME_PATTERNS', [])),
+            })
+
+        return result
+
+    @classmethod
+    def get_profile_info(cls, cui: str) -> Optional[Dict]:
+        """
+        Get detailed info about a profile.
+
+        Args:
+            cui: CUI to lookup
+
+        Returns:
+            Dict with profile details or None
+        """
+        profile = cls.get_profile(cui)
+        if not profile:
+            return None
+
+        return {
+            "cui": cui,
+            "cuis": list(profile.CUI_LIST),
+            "class_name": profile.__class__.__name__,
+            "store_name": profile.STORE_NAME,
+            "name_patterns": list(profile.NAME_PATTERNS),
+            "validation_hints": profile.get_validation_hints(),
+        }
+
+    # -------------------------------------------------------------------------
+    # Hot-Reload
+    # -------------------------------------------------------------------------
+
+    @classmethod
+    def reload_all(cls) -> int:
+        """
+        Hot-reload all profile modules.
+
+        Clears instance cache and reloads all .py files in profiles directory.
+        Decorator re-registers classes with updated code.
+
+        Returns:
+            Number of modules reloaded
+        """
+        logger.info("Starting profile hot-reload...")
+
+        # Clear instance cache (will be recreated on next get_profile)
+        cls._instances.clear()
+
+        # Get list of profile modules (exclude __init__, base)
+        module_names = cls._get_profile_module_names()
+
+        # Determine the module prefix based on how THIS module was imported
+        base_package = cls.__module__
+
+        count = 0
+        for module_name in module_names:
+            full_name = f"{base_package}.{module_name}"
+
+            try:
+                if full_name in sys.modules:
+                    # Reload existing module
+                    importlib.reload(sys.modules[full_name])
+                    logger.debug(f"Reloaded module: {module_name}")
+                else:
+                    # Import new module
+                    importlib.import_module(full_name)
+                    logger.debug(f"Imported new module: {module_name}")
+                count += 1
+            except Exception as e:
+                logger.error(f"Failed to reload {module_name}: {e}")
+
+        cls._last_reload = datetime.utcnow()
+        cls._loaded = True
+
+        logger.info(f"Profile hot-reload complete: {count} modules, {len(cls._profiles)} profiles")
+        return count
+
+    @classmethod
+    def get_reload_status(cls) -> Dict:
+        """Get status of the registry including last reload time."""
+        return {
+            "loaded": cls._loaded,
+            "last_reload": cls._last_reload.isoformat() if cls._last_reload else None,
+            "profiles_count": len(cls._profiles),
+            "instances_count": len(cls._instances),
+            "registered_cuis": list(cls._profiles.keys()),
+        }
+
+    # -------------------------------------------------------------------------
+    # Internal methods
+    # -------------------------------------------------------------------------
+
+    @classmethod
+    def _normalize_cui(cls, cui: str) -> str:
+        """
+        Normalize CUI for consistent lookup.
+
+        - Removes RO prefix (with or without space)
+        - Strips whitespace
+        - Converts to uppercase
+
+        Args:
+            cui: Raw CUI string
+
+        Returns:
+            Normalized CUI (digits only)
+        """
+        if not cui:
+            return ""
+
+        cui = str(cui).strip().upper()
+
+        # Remove RO prefix (handles "RO12345" and "RO 12345")
+        if cui.startswith("RO"):
+            cui = cui[2:].lstrip()
+
+        return cui.strip()
+
+    @classmethod
+    def _get_profile_module_names(cls) -> List[str]:
+        """
+        Get list of profile module names from profiles directory.
+
+        Excludes __init__.py and base.py.
+
+        Returns:
+            List of module names (without .py extension)
+        """
+        excluded = {"__init__", "base", "__pycache__"}
+        modules = []
+
+        for path in PROFILES_DIR.glob("*.py"):
+            name = path.stem
+            if name not in excluded:
+                modules.append(name)
+
+        return sorted(modules)
+
+    @classmethod
+    def _load_all_profiles(cls) -> None:
+        """
+        Initial load of all profile modules.
+
+        Called automatically on first get_profile() if not already loaded.
+        """
+        if cls._loaded:
+            return
+
+        logger.info("Loading store profiles...")
+
+        module_names = cls._get_profile_module_names()
+
+        # Determine the module prefix based on how THIS module was imported
+        # This handles both:
+        # - Running from backend dir: "modules.data_entry.services.ocr.profiles"
+        # - Running from project root: "backend.modules.data_entry.services.ocr.profiles"
+        this_module = cls.__module__  # e.g. "backend.modules..." or "modules..."
+        base_package = this_module  # Use the same prefix for child modules
+
+        for module_name in module_names:
+            full_name = f"{base_package}.{module_name}"
+            try:
+                importlib.import_module(full_name)
+                logger.debug(f"Loaded module: {module_name}")
+            except Exception as e:
+                logger.error(f"Failed to load {module_name}: {e}")
+
+        cls._loaded = True
+        cls._last_reload = datetime.utcnow()
+
+        logger.info(f"Loaded {len(cls._profiles)} store profiles")
+
+    @classmethod
+    def clear(cls) -> None:
+        """
+        Clear all registered profiles.
+
+        Mainly useful for testing.
+        """
+        cls._profiles.clear()
+        cls._instances.clear()
+        cls._loaded = False
+        cls._last_reload = None
+
+
+# -------------------------------------------------------------------------
+# Module exports
+# -------------------------------------------------------------------------
+
+__all__ = [
+    "ProfileRegistry",
+    "BaseStoreProfile",
+]
+
+# Re-export BaseStoreProfile for convenience
+from .base import BaseStoreProfile