Compare commits

14 Commits

Author SHA1 Message Date
Claude Agent
371c73f5cf fix(deploy.ps1): strip non-ASCII so Windows PowerShell parses it
The file was UTF-8 without BOM; Windows PowerShell 5.1 reads it as CP1252,
where the box-drawing/em-dash/arrow bytes (0x94/0x92) decode to smart quotes
(U+201D/U+2019) that PS treats as string delimiters, breaking the parser
("string is missing the terminator"). Replaced -, =, em/en dash, arrows and
smart quotes with ASCII equivalents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:18:55 +00:00
Claude Agent
3b56029cf2 Merge feat/db-log-maintenance: guard DB + log growth (prune + Option B + rotation) 2026-06-26 10:05:18 +00:00
Claude Agent
dcc5042586 feat(maintenance): guard DB + log growth (Option B + daily prune + rotation)
Root cause of the 2GB prod import.db: the sync_run_orders audit junction
recorded every order on every run; under the 1-minute scheduler ~98% of
21.7M rows were no-op ALREADY_IMPORTED re-observations. NSSM stdout/stderr
also grew unbounded (rotation never applied to the live service).

Changes:
- sqlite_service: skip ALREADY_IMPORTED rows in sync_run_orders (write-side
  guard, _SKIP_JUNCTION_STATUSES); add prune_sync_history(retention_days)
  with incremental_vacuum.
- maintenance_service (new): cleanup_old_logs + run_daily_maintenance.
- scheduler_service: start_maintenance_job (daily CronTrigger).
- main.py: RotatingFileHandler (sync_comenzi_current.log, 10MB x5) instead
  of a new timestamped file per start; schedule daily maintenance + one-shot
  catch-up at startup.
- scripts/db_maintenance.py (new): one-shot prune + VACUUM + log cleanup,
  plain sqlite3, invoked by deploy.ps1 while the service is stopped.
- deploy.ps1: stop -> run db_maintenance.py -> (re)apply NSSM AppRotate*
  idempotently -> start, so rotation reaches pre-existing services.

Retention defaults: 7 days history, 7 days logs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:59:41 +00:00
Claude Agent
ccc6a933fa feat(scripts): line-level residual check in relink_manual_invoices
Linking the VANZARI header (ID_COMANDA) makes the app dashboard show an
order facturat, but ROA decides facturat at the LINE level
(PACK_FACTURARE.cursor_comanda matches invoiced qty on ID_ARTICOL + exact
PRET). When a manual invoice represented lines differently than the order
(e.g. per-VAT-rate discounts consolidated into one 0%-TVA line), the order
stays nefacturat in ROA despite the header link.

Add order_line_residual(): predicts the residual before --apply (via
extra_idv) and re-verifies after linking. Warns in the plan, the summary
counter, and post-apply when an order will still show nefacturat. The
script never touches COMENZI_ELEMENTE — those need a manual line fix.

Surfaced by orders 5419/5423 (web 492710430/492710513) on 2026-06-26.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:46:59 +00:00
Claude Agent
698d036de9 docs: ghid reconciliere facturi manuale ROA <-> comenzi GoMag
Documenteaza relink_manual_invoices.py/.bat: cand se foloseste, cauza tehnica
(VANZARI.ID_COMANDA NULL), logica de matching conservatoare, clasificarea
LINK/SKIP_*, exceptia facturilor de depozit, si workflow-ul de rulare pe prod.
Link in tabelul Documentatie Tehnica din README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:37:19 +00:00
Claude Agent
5dd5acc25e Merge feat/relink-manual-invoices: tool to reconcile manual ROA invoices with GoMag orders 2026-06-26 08:35:07 +00:00
Claude Agent
df684b7183 feat(scripts): add Windows .bat launcher for relink_manual_invoices
Sets the Oracle thick-mode env (TNS_ADMIN + instant client PATH) like start.bat,
resolves venv/script paths relative to itself, forwards all args, and pauses so
output is readable. Run on the VENDING prod server (double-click = dry-run).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:34:15 +00:00
Claude Agent
2c3b35294c feat(scripts): reconcile manual ROA invoices with GoMag orders
Tool to relink orphan VANZARI invoices (ID_COMANDA NULL, sters=0) — emitted
manually by the warehouse during an Oracle/sync outage — to their GoMag order,
then populate the SQLite invoice cache like the app does.

Matching is conservative (exact total + partner OR name; handles duplicate
partner records) and only auto-links unambiguous 1:1 matches. Genuine warehouse/
walk-in invoices (no online order behind them) get SKIP_NOMATCH and are left
untouched; anything unclear is reported SKIP_AMBIGUOUS for manual review.

Dry-run by default; --apply / --yes / --days N. Runs on prod (uses app.config
settings + prod import.db). Codifies the 2026-06-26 manual relink (12 invoices).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:13:02 +00:00
Claude Agent
19834d193a Merge fix/roa-mass-deletion-guard: ROA mass-deletion guard + Oracle pool auto-recovery
- Guard against falsely mass-marking DELETED_IN_ROA when ROA is recovering
- Oracle pool self-heals per sync cycle; status surfaced; order_detail no longer 500s

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:41:21 +00:00
Claude Agent
cd7eb628dd feat(oracle): auto-recover Oracle pool + surface status, stop silent import failures
After a power loss the app started before Oracle was ready; init_oracle() failed
once, the pool stayed None forever (no retry), and every sync silently failed
("Oracle pool not initialized") while still hammering the GoMag API each minute,
and order-detail 500'd.

- database.ensure_oracle_pool(force): thread-safe (re)create of the pool, called
  at the start of every sync cycle → self-heals within one cycle once Oracle is
  back (incl. after an Oracle service restart). init_oracle_client made idempotent
  so re-init can't fall back to thin mode.
- database.oracle_status() exposed; main.py startup is non-fatal via ensure pool.
- run_sync ensures the pool before the GoMag download; on failure it records a
  clear run status instead of crashing and skips the wasted API calls.
- /api/sync/health reports oracle_ready/last_error; dashboard health pill shows
  "Oracle indisponibil" (top priority). Recovery via the existing Start Sync button.
- order_detail degrades gracefully (200 without CODMAT + notice) instead of 500.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:33:01 +00:00
Claude Agent
395e2b997a fix(sync): guard against mass DELETED_IN_ROA when ROA is recovering
After a power loss + reboot, COMENZI was queryable but not yet recovered;
phase 4b-3 read it as empty and sticky-marked 3794 live orders DELETED_IN_ROA
(nulling id_comanda). check_orders_exist also swallowed Oracle errors and
returned a partial set, which callers misread as deletions.

- check_orders_exist now re-raises on Oracle error instead of returning partial
- new invoice_service.deletions_or_guard() raises MassDeletionGuard when the
  would-delete fraction is implausibly high (>30% of >=25 imported orders)
- both deletion sites (auto sync + manual refresh) skip + log on guard trip

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 07:18:08 +00:00
Claude Agent
4a03fe1016 docs: ghid cautare/selectie client dupa cod fiscal + verificare ANAF
Document de referinta end-to-end (replicabil in alt proiect): normalizare
CUI (strip RO, typo OCR, checksum), client ANAF (v9/tva, contract tri-valent,
cache), gate CUI, mod strict platitor/neplatitor TVA si procedurile Oracle
cauta_partener_dupa_cod_fiscal / cauta_sau_creeaza_partener. Link in README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 06:26:38 +00:00
Claude Agent
71a3b32bd7 Merge fix/mappings-slash-404: query params pentru sku/codmat cu slash 2026-06-08 14:01:19 +00:00
Claude Agent
550d94ff16 fix(mappings): move sku/codmat from path to query — slash in keys 404'd
Codurile ROA contin legitim '/' si '"' (ex. RCR1/4"). Rutele FastAPI
foloseau parametri de cale (/api/mappings/{sku}/{codmat}...); ASGI decodeaza
%2F -> '/' INAINTE de routing, deci o cheie cu slash devine doua segmente si
Starlette nu mai potriveste ruta -> 404 la edit/delete/restore. Cand ambii
parametri au slash, un convertor {path} nu rezolva (split ambiguu).

Solutie: sku/codmat trec din cale in query string. %2F/%22 din query se
decodeaza ca parte a valorii, fara sa atinga routing-ul de cale.

- backend: 4 rute statice + Query(...) (update/edit/delete/restore);
  semnaturile mapping_service raman neschimbate
- frontend: helper mapQS(); toate cele 8 apeluri path -> query; cache-bust ?v=19
- tests: regresie ci-layer (chei-sentinel cu slash, !=404, toate 4 rutele);
  test_integration update/delete convertite la query; e2e delete RCR1/4"
  (slash+ghilimea) prin modalul UI, skip cand Oracle lipseste

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 13:58:04 +00:00
26 changed files with 1654 additions and 119 deletions

View File

@@ -516,6 +516,8 @@ ssh -i ~/.ssh/id_ed25519 -p 22122 gomag@79.119.86.134 \
|--------|---------|
| [docs/oracle-schema-notes.md](docs/oracle-schema-notes.md) | Schema Oracle: tabele comenzi, facturi, preturi, proceduri cheie |
| [docs/pack_facturare_analysis.md](docs/pack_facturare_analysis.md) | Analiza flow facturare: call chain, parametri, STOC lookup, FACT-008 |
| [docs/cautare-selectie-client-cod-fiscal-anaf.md](docs/cautare-selectie-client-cod-fiscal-anaf.md) | Cautare/selectie client dupa cod fiscal: normalizare CUI, verificare ANAF, gate, proceduri Oracle |
| [docs/relink-facturi-manuale.md](docs/relink-facturi-manuale.md) | Reconciliere facturi manuale ROA ↔ comenzi GoMag dupa downtime (`relink_manual_invoices.py/.bat`) |
| [scripts/HANDOFF_MAPPING.md](scripts/HANDOFF_MAPPING.md) | Matching GoMag SKU → ROA articole (strategie si rezultate) |
---

View File

@@ -3,39 +3,59 @@ import aiosqlite
import sqlite3
import logging
import os
import threading
from datetime import datetime
from .config import settings
logger = logging.getLogger(__name__)
# ---- Oracle Pool ----
pool = None
_pool_lock = threading.Lock()
_pool_last_error = None # str — reason the last (re)init failed, or None
_pool_last_attempt = None # ISO str — when we last tried to (re)init
_client_initialized = False # init_oracle_client may only be called once/process
def init_oracle():
"""Initialize Oracle client mode and create connection pool."""
global pool
def _init_oracle_client_once():
"""Load the Oracle client library exactly once.
init_oracle_client() loads the thick-mode driver (it does NOT connect to the
DB), so it succeeds even when Oracle is down. Calling it a second time raises,
which on a pool re-init would wrongly fall back to thin mode — so we guard it.
"""
global _client_initialized
if _client_initialized:
return
force_thin = settings.FORCE_THIN_MODE
instantclient_path = settings.INSTANTCLIENTPATH
dsn = settings.ORACLE_DSN
# Ensure TNS_ADMIN is set as OS env var so oracledb can find tnsnames.ora
if settings.TNS_ADMIN:
os.environ['TNS_ADMIN'] = settings.TNS_ADMIN
logger.info(f"Oracle config: DSN={dsn}, TNS_ADMIN={settings.TNS_ADMIN or os.environ.get('TNS_ADMIN', '(not set)')}, INSTANTCLIENTPATH={instantclient_path or '(not set)'}")
logger.info(f"Oracle config: DSN={settings.ORACLE_DSN}, TNS_ADMIN={settings.TNS_ADMIN or os.environ.get('TNS_ADMIN', '(not set)')}, INSTANTCLIENTPATH={instantclient_path or '(not set)'}")
if force_thin:
logger.info(f"FORCE_THIN_MODE=true: thin mode for {dsn}")
logger.info(f"FORCE_THIN_MODE=true: thin mode for {settings.ORACLE_DSN}")
elif instantclient_path:
try:
oracledb.init_oracle_client(lib_dir=instantclient_path)
logger.info(f"Thick mode activated for {dsn}")
logger.info(f"Thick mode activated for {settings.ORACLE_DSN}")
except Exception as e:
logger.error(f"Thick mode error: {e}")
logger.info("Fallback to thin mode")
else:
logger.info(f"Thin mode (default) for {dsn}")
logger.info(f"Thin mode (default) for {settings.ORACLE_DSN}")
_client_initialized = True
def init_oracle():
"""Initialize Oracle client mode and create the connection pool. Raises on failure."""
global pool
_init_oracle_client_once()
pool = oracledb.create_pool(
user=settings.ORACLE_USER,
password=settings.ORACLE_PASSWORD,
@@ -44,9 +64,49 @@ def init_oracle():
max=4,
increment=1
)
logger.info(f"Oracle pool created for {dsn}")
logger.info(f"Oracle pool created for {settings.ORACLE_DSN}")
return pool
def ensure_oracle_pool(force: bool = False) -> bool:
"""Ensure the Oracle pool exists, (re)creating it if needed. Returns True if ready.
Thread-safe and idempotent — safe to call at the start of every sync cycle so
the app self-heals after Oracle becomes reachable again (e.g. the DB service
was restarted after a power loss). On failure it records the reason and leaves
pool=None so callers can surface a clear status instead of crashing.
"""
global pool, _pool_last_error, _pool_last_attempt
with _pool_lock:
if pool is not None and not force:
return True
if force and pool is not None:
try:
pool.close()
except Exception:
pass
pool = None
_pool_last_attempt = datetime.now().isoformat()
try:
init_oracle()
_pool_last_error = None
return True
except Exception as e:
pool = None
_pool_last_error = str(e)
logger.error(f"Oracle pool init failed: {e}")
return False
def oracle_status() -> dict:
"""Snapshot of Oracle pool readiness for health endpoints."""
return {
"ready": pool is not None,
"last_error": _pool_last_error,
"last_attempt_at": _pool_last_attempt,
}
def get_oracle_connection():
"""Get a connection from the Oracle pool."""
if pool is None:

View File

@@ -1,13 +1,14 @@
import asyncio
from contextlib import asynccontextmanager
from datetime import datetime
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from pathlib import Path
import logging
import logging.handlers
import os
from .config import settings
from .database import init_oracle, close_oracle, init_sqlite
from .database import ensure_oracle_pool, close_oracle, init_sqlite
# Configure logging with both stream and file handlers
_log_level = getattr(logging, settings.LOG_LEVEL.upper(), logging.INFO)
@@ -19,8 +20,12 @@ _stream_handler.setFormatter(_formatter)
_log_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'logs')
os.makedirs(_log_dir, exist_ok=True)
_log_filename = f"sync_comenzi_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log"
_file_handler = logging.FileHandler(os.path.join(_log_dir, _log_filename), encoding='utf-8')
# Rotating handler (10MB x 5 backups) instead of a new timestamped file per
# start — caps log growth and stops file proliferation across restarts. Fixed
# name still matches the QA glob `sync_comenzi_*.log`.
_file_handler = logging.handlers.RotatingFileHandler(
os.path.join(_log_dir, "sync_comenzi_current.log"),
maxBytes=10 * 1024 * 1024, backupCount=5, encoding='utf-8')
_file_handler.setFormatter(_formatter)
_root_logger = logging.getLogger()
@@ -35,12 +40,10 @@ async def lifespan(app: FastAPI):
"""Startup and shutdown events."""
logger.info("Starting GoMag Import Manager...")
# Initialize Oracle pool
try:
init_oracle()
except Exception as e:
logger.error(f"Oracle init failed: {e}")
# Allow app to start even without Oracle for development
# Initialize Oracle pool (non-fatal: app still starts if Oracle is down;
# each sync cycle calls ensure_oracle_pool() and self-heals when it returns)
if not ensure_oracle_pool():
logger.error("Oracle pool not ready at startup — will retry on each sync cycle")
# Initialize SQLite
init_sqlite()
@@ -56,6 +59,15 @@ async def lifespan(app: FastAPI):
except Exception:
pass
# Daily DB/log maintenance (prune audit history + cleanup old logs) + a
# one-shot catch-up so a long-down service reclaims immediately on start.
try:
from .services import maintenance_service
scheduler_service.start_maintenance_job()
asyncio.create_task(maintenance_service.run_daily_maintenance())
except Exception as e:
logger.warning(f"Maintenance scheduling failed: {e}")
logger.info("GoMag Import Manager started")
yield

View File

@@ -95,16 +95,18 @@ async def create_mapping(data: MappingCreate):
except Exception as e:
return {"success": False, "error": str(e)}
@router.put("/api/mappings/{sku}/{codmat}")
def update_mapping(sku: str, codmat: str, data: MappingUpdate):
# sku/codmat in query string (not path) — codurile ROA contin legitim '/' si '"'
# (ex. RCR1/4"); %2F in path e decodat de ASGI inainte de routing -> 404.
@router.put("/api/mappings/update")
def update_mapping(data: MappingUpdate, sku: str = Query(...), codmat: str = Query(...)):
try:
updated = mapping_service.update_mapping(sku, codmat, data.cantitate_roa, data.activ)
return {"success": updated}
except Exception as e:
return {"success": False, "error": str(e)}
@router.put("/api/mappings/{sku}/{codmat}/edit")
def edit_mapping(sku: str, codmat: str, data: MappingEdit):
@router.put("/api/mappings/edit")
def edit_mapping(data: MappingEdit, sku: str = Query(...), codmat: str = Query(...)):
try:
result = mapping_service.edit_mapping(sku, codmat, data.new_sku, data.new_codmat,
data.cantitate_roa)
@@ -112,16 +114,16 @@ def edit_mapping(sku: str, codmat: str, data: MappingEdit):
except Exception as e:
return {"success": False, "error": str(e)}
@router.delete("/api/mappings/{sku}/{codmat}")
def delete_mapping(sku: str, codmat: str):
@router.delete("/api/mappings/delete")
def delete_mapping(sku: str = Query(...), codmat: str = Query(...)):
try:
deleted = mapping_service.delete_mapping(sku, codmat)
return {"success": deleted}
except Exception as e:
return {"success": False, "error": str(e)}
@router.post("/api/mappings/{sku}/{codmat}/restore")
def restore_mapping(sku: str, codmat: str):
@router.post("/api/mappings/restore")
def restore_mapping(sku: str = Query(...), codmat: str = Query(...)):
try:
restored = mapping_service.restore_mapping(sku, codmat)
return {"success": restored}

View File

@@ -190,8 +190,11 @@ async def sync_health():
counts = await sqlite_service.get_recent_phase_failures(limit=3)
escalation_phase = next((p for p, c in counts.items() if c >= 3), None)
ora = database.oracle_status()
is_healthy = (
last_status in (None, "completed")
ora["ready"]
and last_status in (None, "completed")
and escalation_phase is None
and sum(counts.values()) <= 1
)
@@ -203,6 +206,9 @@ async def sync_health():
"recent_phase_failures": counts,
"escalation_phase": escalation_phase,
"is_healthy": is_healthy,
"oracle_ready": ora["ready"],
"oracle_last_error": ora["last_error"],
"oracle_last_attempt_at": ora["last_attempt_at"],
}
@@ -422,10 +428,18 @@ async def order_detail(order_number: str):
return {"error": "Order not found"}
items = detail.get("items", [])
oracle_available = True
try:
await _enrich_items_with_codmat(items)
except Exception as e:
# Oracle down (pool not initialized): still return the order with its
# items so the detail panel renders, just without CODMAT enrichment.
oracle_available = False
logger.warning(f"order_detail CODMAT enrich skipped (Oracle unavailable?): {e}")
# Enrich with invoice data
order = detail.get("order", {})
order["oracle_available"] = oracle_available
if order.get("factura_numar") and order.get("factura_data"):
order["invoice"] = {
"facturat": True,
@@ -846,8 +860,12 @@ async def refresh_invoices():
existing_ids = await asyncio.to_thread(
invoice_service.check_orders_exist, id_comanda_list
)
for o in all_imported:
if o["id_comanda"] not in existing_ids:
try:
to_delete = invoice_service.deletions_or_guard(all_imported, existing_ids)
except invoice_service.MassDeletionGuard as g:
logger.warning(f"Mass-deletion guard tripped during refresh: {g}")
to_delete = []
for o in to_delete:
await sqlite_service.mark_order_deleted_in_roa(o["order_number"])
orders_deleted += 1

View File

@@ -3,6 +3,39 @@ from .. import database
logger = logging.getLogger(__name__)
# ── Mass-deletion safety guard ──────────────────────────────────────────────
# If ROA appears to have lost a large fraction of its orders, it is almost
# certainly a transient/recovery state (e.g. the DB just restarted after a power
# loss and COMENZI hasn't finished recovering), NOT real deletions. In that case
# we refuse to mass-mark orders as DELETED_IN_ROA — a sticky, hard-to-reverse
# operation that nulls id_comanda. See incident 2026-06-26 (3794 false deletes).
MASS_DELETION_ABORT_FRACTION = 0.30
MASS_DELETION_ABORT_MIN = 25
class MassDeletionGuard(Exception):
"""Raised when the number of orders that would be marked deleted is
suspiciously high, indicating ROA is unavailable rather than truly purged."""
def deletions_or_guard(all_imported: list, existing_ids: set) -> list:
"""Return the subset of all_imported whose id_comanda is missing from ROA,
or raise MassDeletionGuard if that subset is implausibly large.
`existing_ids` MUST come from a successful check_orders_exist call — that
function now raises on Oracle error rather than returning a partial set, so
an empty result here means ROA genuinely has none of these orders.
"""
missing = [o for o in all_imported if o["id_comanda"] not in existing_ids]
total = len(all_imported)
if total >= MASS_DELETION_ABORT_MIN and len(missing) > total * MASS_DELETION_ABORT_FRACTION:
raise MassDeletionGuard(
f"{len(missing)}/{total} comenzi par sterse din ROA "
f"(>{int(MASS_DELETION_ABORT_FRACTION * 100)}%) — posibil ROA "
f"indisponibil/in recuperare; marcarea DELETED_IN_ROA a fost ANULATA"
)
return missing
def check_invoices_for_orders(id_comanda_list: list) -> dict:
"""Check which orders have been invoiced in Oracle (vanzari table).
@@ -68,7 +101,11 @@ def check_orders_exist(id_comanda_list: list) -> set:
for row in cur:
existing.add(row[0])
except Exception as e:
# Do NOT swallow: a partial/empty result on error would be misread by
# callers as "these orders were deleted in ROA" and trigger sticky
# DELETED_IN_ROA marking. Propagate so the caller skips deletion.
logger.warning(f"Order existence check failed: {e}")
raise
finally:
database.pool.release(conn)

View File

@@ -0,0 +1,75 @@
"""Periodic maintenance: prune audit history + clean up old log files.
Keeps the SQLite DB and the logs/ directory from growing unbounded. The audit
tables (sync_runs, sync_run_orders) were the only DB growth source under the
1-minute scheduler; business tables (orders, order_items) are never touched.
The one-shot heavy reclaim (full VACUUM, run while the service is stopped) lives
in scripts/db_maintenance.py and is invoked by deploy.ps1.
"""
import logging
import os
import time
logger = logging.getLogger(__name__)
DEFAULT_HISTORY_RETENTION_DAYS = 7
DEFAULT_LOG_RETENTION_DAYS = 7
def _logs_dir() -> str:
"""Absolute path to the repo-root logs/ directory (matches main.py)."""
here = os.path.dirname(os.path.abspath(__file__))
return os.path.join(os.path.abspath(os.path.join(here, "..", "..", "..")), "logs")
def cleanup_old_logs(retention_days: int = DEFAULT_LOG_RETENTION_DAYS,
log_dir: str | None = None) -> int:
"""Delete log files older than `retention_days`. Returns count removed.
Targets any file with `.log` in its name (covers `sync_comenzi_current.log`,
NSSM `service_stdout.log`, and rotated backups like `*.log.3`). The live
rotating files stay fresh (recent mtime) so they fall inside the window.
"""
log_dir = log_dir or _logs_dir()
if not os.path.isdir(log_dir):
return 0
cutoff = time.time() - retention_days * 86400
removed = 0
for name in os.listdir(log_dir):
if ".log" not in name:
continue
path = os.path.join(log_dir, name)
try:
if os.path.isfile(path) and os.path.getmtime(path) < cutoff:
os.remove(path)
removed += 1
except OSError as e:
logger.warning(f"cleanup_old_logs: could not remove {name}: {e}")
if removed:
logger.info(f"cleanup_old_logs: removed {removed} file(s) older than "
f"{retention_days}d from {log_dir}")
return removed
async def run_daily_maintenance(
history_days: int = DEFAULT_HISTORY_RETENTION_DAYS,
log_days: int = DEFAULT_LOG_RETENTION_DAYS) -> dict:
"""Daily job: prune audit history (+reclaim pages) and clean old log files.
Each step is isolated — a failure in one does not skip the other.
"""
from . import sqlite_service
result: dict = {}
try:
result["db"] = await sqlite_service.prune_sync_history(history_days)
except Exception as e:
logger.warning(f"run_daily_maintenance: prune_sync_history failed: {e}")
result["db_error"] = str(e)
try:
result["logs_removed"] = cleanup_old_logs(log_days)
except Exception as e:
logger.warning(f"run_daily_maintenance: cleanup_old_logs failed: {e}")
result["logs_error"] = str(e)
return result

View File

@@ -1,6 +1,7 @@
import logging
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.interval import IntervalTrigger
from apscheduler.triggers.cron import CronTrigger
logger = logging.getLogger(__name__)
@@ -42,6 +43,31 @@ def start_scheduler(interval_minutes: int = 10):
logger.info(f"Scheduler started with interval {interval_minutes}min")
def start_maintenance_job(hour: int = 3):
"""Schedule the daily DB/log maintenance job (prune history + cleanup logs).
Runs independently of the sync job — starts the scheduler if it isn't already
running so maintenance happens even when auto-sync is disabled.
"""
if _scheduler is None:
init_scheduler()
from . import maintenance_service
_scheduler.add_job(
maintenance_service.run_daily_maintenance,
trigger=CronTrigger(hour=hour, minute=0),
id="maintenance_job",
name="Daily DB/Log Maintenance",
replace_existing=True
)
if not _scheduler.running:
_scheduler.start()
logger.info(f"Maintenance job scheduled daily at {hour:02d}:00")
def stop_scheduler():
"""Stop the scheduler."""
global _is_running

View File

@@ -2,7 +2,7 @@ import json
import logging
import logging.handlers
import os
from datetime import datetime
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
from ..database import get_sqlite, get_sqlite_sync
from ..constants import OrderStatus
@@ -114,6 +114,45 @@ async def update_sync_run(run_id: str, status: str, total_orders: int = 0,
await db.close()
async def prune_sync_history(retention_days: int = 7) -> dict:
"""Delete sync_runs + sync_run_orders older than `retention_days`.
Audit-only tables — `orders`/`order_items` (business data) are never touched.
Frees pages via incremental_vacuum (prod DB is auto_vacuum=INCREMENTAL after
the initial reclaim). Returns counts for logging. See _SKIP_JUNCTION_STATUSES
for the complementary write-side guard.
"""
cutoff = (datetime.now(_tz_bucharest).replace(tzinfo=None)
- timedelta(days=retention_days)).strftime("%Y-%m-%d")
db = await get_sqlite()
try:
cur = await db.execute(
"DELETE FROM sync_run_orders WHERE sync_run_id IN "
"(SELECT run_id FROM sync_runs WHERE substr(started_at,1,10) < ?)",
(cutoff,))
junction_deleted = cur.rowcount
cur = await db.execute(
"DELETE FROM sync_runs WHERE substr(started_at,1,10) < ?", (cutoff,))
runs_deleted = cur.rowcount
# Drop phase-failure rows orphaned by the run deletion.
await db.execute(
"DELETE FROM sync_phase_failures "
"WHERE run_id NOT IN (SELECT run_id FROM sync_runs)")
await db.commit()
try:
await db.execute("PRAGMA incremental_vacuum")
await db.commit()
except Exception as e: # auto_vacuum may be OFF on a fresh dev DB
logger.debug(f"prune_sync_history: incremental_vacuum skipped: {e}")
logger.info(
f"prune_sync_history: cutoff<{cutoff} runs_deleted={runs_deleted} "
f"junction_deleted={junction_deleted}")
return {"cutoff": cutoff, "runs_deleted": runs_deleted,
"junction_deleted": junction_deleted}
finally:
await db.close()
async def upsert_order(sync_run_id: str, order_number: str, order_date: str,
customer_name: str, status: str, id_comanda: int = None,
id_partener: int = None, error_message: str = None,
@@ -171,8 +210,28 @@ async def upsert_order(sync_run_id: str, order_number: str, order_date: str,
await db.close()
# Audit junction policy (DB-size guard):
# The sync_run_orders junction recorded EVERY order seen on EVERY run. Under the
# 1-minute scheduler, ~98% of rows were no-op ALREADY_IMPORTED re-observations,
# which grew the table to 21M+ rows / 2GB. We no longer record those: the order's
# current state still lives in `orders`; the junction now only lists orders a run
# actually touched (new / changed / skipped / errored / cancelled). Run-detail
# views therefore show only meaningful orders per run.
_SKIP_JUNCTION_STATUSES = {OrderStatus.ALREADY_IMPORTED.value}
def _record_in_junction(status_at_run: str) -> bool:
"""Whether this per-run status is worth persisting in sync_run_orders."""
return status_at_run not in _SKIP_JUNCTION_STATUSES
async def add_sync_run_order(sync_run_id: str, order_number: str, status_at_run: str):
"""Record that this run processed this order (junction table)."""
"""Record that this run processed this order (junction table).
No-op ALREADY_IMPORTED observations are skipped — see _SKIP_JUNCTION_STATUSES.
"""
if not _record_in_junction(status_at_run):
return
db = await get_sqlite()
try:
await db.execute("""
@@ -258,9 +317,15 @@ async def _insert_orders_only(db, orders: list[dict]):
if not orders:
return
await db.executemany(_ORDERS_UPSERT_SQL, [_orders_row(d) for d in orders])
junction_rows = [
(d["sync_run_id"], d["order_number"], d.get("status_at_run", d["status"]))
for d in orders
if _record_in_junction(d.get("status_at_run", d["status"]))
]
if junction_rows:
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
[(d["sync_run_id"], d["order_number"], d.get("status_at_run", d["status"])) for d in orders],
junction_rows,
)
@@ -273,9 +338,15 @@ async def _insert_valid_batch(db, orders: list[dict]):
if not orders:
return
await db.executemany(_ORDERS_UPSERT_SQL, [_orders_row(d) for d in orders])
junction_rows = [
(d["sync_run_id"], d["order_number"], d["status_at_run"])
for d in orders
if _record_in_junction(d["status_at_run"])
]
if junction_rows:
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
[(d["sync_run_id"], d["order_number"], d["status_at_run"]) for d in orders],
junction_rows,
)
all_items: list[tuple] = []
@@ -314,6 +385,7 @@ async def _insert_single_order(db, d: dict):
Caller wraps in SAVEPOINT so a per-row failure doesn't poison the batch.
"""
await db.execute(_ORDERS_UPSERT_SQL, _orders_row(d))
if _record_in_junction(d["status_at_run"]):
await db.execute(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
(d["sync_run_id"], d["order_number"], d["status_at_run"]),

View File

@@ -338,6 +338,26 @@ async def run_sync(id_pol: int = None, id_sectie: int = None, run_id: str = None
return {"run_id": run_id, "status": "halted_escalation", "error": halt_msg}
try:
# Phase -1: Ensure Oracle pool (auto-recovery after a DB restart).
# Done before the GoMag download so we don't waste API calls every
# cycle while Oracle is down, and so users get a clear status.
if not await asyncio.to_thread(database.ensure_oracle_pool):
last_err = database.oracle_status().get("last_error") or "fara detalii"
msg = ("Oracle indisponibil — pool neinitializat. Import oprit; "
"se reincearca automat la urmatorul ciclu de sync. "
f"Detalii: {last_err}")
_log_line(run_id, f"EROARE: {msg}")
await sqlite_service.create_sync_run(run_id, 0)
await sqlite_service.update_sync_run(
run_id, "failed", 0, 0, 0, 0, error_message=msg
)
if _current_sync:
_current_sync["status"] = "failed"
_current_sync["finished_at"] = _now().isoformat()
_current_sync["error"] = msg
_update_progress("failed", "Oracle indisponibil — import oprit")
return {"run_id": run_id, "status": "failed", "error": msg}
# Phase 0: Download orders from GoMag API
_update_progress("downloading", "Descărcare comenzi din GoMag API...")
_log_line(run_id, "Descărcare comenzi din GoMag API...")
@@ -1081,8 +1101,13 @@ async def run_sync(id_pol: int = None, id_sectie: int = None, run_id: str = None
existing_ids = await asyncio.to_thread(
invoice_service.check_orders_exist, id_comanda_list
)
for o in all_imported:
if o["id_comanda"] not in existing_ids:
try:
to_delete = invoice_service.deletions_or_guard(all_imported, existing_ids)
except invoice_service.MassDeletionGuard as g:
_log_line(run_id, f"⚠ Protectie stergeri: {g}")
await _record_phase_err(run_id, "mass_deletion_guard", g)
to_delete = []
for o in to_delete:
await sqlite_service.mark_order_deleted_in_roa(o["order_number"])
orders_deleted += 1

View File

@@ -184,7 +184,14 @@ function renderHealthPill(h) {
const recent = h.recent_phase_failures || {};
const recentCount = Object.values(recent).reduce((a, b) => a + (b || 0), 0);
if (h.escalation_phase || h.last_sync_status === 'halted_escalation') {
if (h.oracle_ready === false) {
state = 'escalated';
iconCls = 'bi-database-x';
text = 'Oracle indisponibil';
tooltip = `Oracle indisponibil — importurile sunt oprite.\n`
+ `${h.oracle_last_error || ''}\n`
+ `Se reincearca automat la urmatorul sync. Apasa Start Sync pentru a reincerca acum.`;
} else if (h.escalation_phase || h.last_sync_status === 'halted_escalation') {
state = 'escalated';
iconCls = 'bi-x-octagon-fill';
text = 'Blocat';

View File

@@ -8,6 +8,12 @@ let editingMapping = null; // {sku, codmat} when editing
const kitPriceCache = new Map();
// sku/codmat in query string — codurile ROA contin legitim '/' si '"' (ex. RCR1/4");
// %2F in path e decodat de ASGI inainte de routing -> 404.
function mapQS(sku, codmat) {
return `sku=${encodeURIComponent(sku)}&codmat=${encodeURIComponent(codmat)}`;
}
// Load on page ready
document.addEventListener('DOMContentLoaded', () => {
loadMappings();
@@ -242,7 +248,7 @@ function editFlatValue(span, sku, codmat, field, currentValue) {
try {
const body = {};
body[field] = newValue;
const res = await fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}`, {
const res = await fetch(`/api/mappings/update?${mapQS(sku, codmat)}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
@@ -432,7 +438,7 @@ async function saveMapping() {
if (editingMapping) {
if (mappings.length === 1) {
// Single CODMAT edit: use existing PUT endpoint
res = await fetch(`/api/mappings/${encodeURIComponent(editingMapping.sku)}/${encodeURIComponent(editingMapping.codmat)}/edit`, {
res = await fetch(`/api/mappings/edit?${mapQS(editingMapping.sku, editingMapping.codmat)}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
@@ -450,7 +456,7 @@ async function saveMapping() {
// Delete each existing CODMAT for old SKU
for (const m of existing) {
await fetch(`/api/mappings/${encodeURIComponent(m.sku)}/${encodeURIComponent(m.codmat)}`, {
await fetch(`/api/mappings/delete?${mapQS(m.sku, m.codmat)}`, {
method: 'DELETE'
});
}
@@ -588,7 +594,7 @@ function cancelInlineAdd() {
async function toggleActive(sku, codmat, currentActive) {
const newActive = currentActive ? 0 : 1;
try {
const res = await fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}`, {
const res = await fetch(`/api/mappings/update?${mapQS(sku, codmat)}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ activ: newActive })
@@ -601,7 +607,7 @@ async function toggleActive(sku, codmat, currentActive) {
// Show toast with undo
const action = newActive ? 'activata' : 'dezactivata';
showUndoToast(`Mapare ${sku} \u2192 ${codmat} ${action}.`, () => {
fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}`, {
fetch(`/api/mappings/update?${mapQS(sku, codmat)}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ activ: currentActive })
@@ -643,7 +649,7 @@ function initDeleteModal() {
if (!pendingDelete) return;
const { sku, codmat } = pendingDelete;
try {
const res = await fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}`, {
const res = await fetch(`/api/mappings/delete?${mapQS(sku, codmat)}`, {
method: 'DELETE'
});
const data = await res.json();
@@ -669,7 +675,7 @@ function deleteMappingConfirm(sku, codmat) {
async function restoreMapping(sku, codmat) {
try {
const res = await fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}/restore`, {
const res = await fetch(`/api/mappings/restore?${mapQS(sku, codmat)}`, {
method: 'POST'
});
const data = await res.json();
@@ -721,7 +727,7 @@ function handleMappingConflict(data) {
const sku = (document.getElementById('inlineSku') || document.getElementById('inputSku'))?.value?.trim();
const codmat = (document.getElementById('inlineCodmat') || document.querySelector('.cl-codmat'))?.value?.trim();
if (sku && codmat) {
fetch(`/api/mappings/${encodeURIComponent(sku)}/${encodeURIComponent(codmat)}/restore`, { method: 'POST' })
fetch(`/api/mappings/restore?${mapQS(sku, codmat)}`, { method: 'POST' })
.then(r => r.json())
.then(d => {
if (d.success) { cancelInlineAdd(); loadMappings(); }

View File

@@ -865,9 +865,13 @@ async function renderOrderDetailModal(orderNumber, opts) {
// Render compact header info (partner + addresses)
_renderHeaderInfo(order);
if (order.error_message) {
document.getElementById('detailError').textContent = order.error_message;
document.getElementById('detailError').style.display = '';
const detailErrEl = document.getElementById('detailError');
if (order.oracle_available === false) {
detailErrEl.textContent = '⚠ Oracle indisponibil — CODMAT-urile nu pot fi incarcate momentan. Reincearca dupa restabilirea conexiunii.';
detailErrEl.style.display = '';
} else if (order.error_message) {
detailErrEl.textContent = order.error_message;
detailErrEl.style.display = '';
}
// Configure footer action buttons BEFORE any early-return on items —

View File

@@ -169,7 +169,7 @@
<script>window.ROOT_PATH = "{{ rp }}";</script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js"></script>
<script src="{{ rp }}/static/js/shared.js?v=50"></script>
<script src="{{ rp }}/static/js/shared.js?v=51"></script>
<script>
// Dark mode toggle
function toggleDarkMode() {

View File

@@ -121,5 +121,5 @@
{% endblock %}
{% block scripts %}
<script src="{{ request.scope.get('root_path', '') }}/static/js/dashboard.js?v=52"></script>
<script src="{{ request.scope.get('root_path', '') }}/static/js/dashboard.js?v=53"></script>
{% endblock %}

View File

@@ -159,5 +159,5 @@
{% endblock %}
{% block scripts %}
<script src="{{ request.scope.get('root_path', '') }}/static/js/mappings.js?v=18"></script>
<script src="{{ request.scope.get('root_path', '') }}/static/js/mappings.js?v=19"></script>
{% endblock %}

View File

@@ -0,0 +1,78 @@
"""E2E regression: delete a mapping whose SKU+CODMAT contain '/' and '"'.
Covers the full real path: render → onclick="...jsAttrEsc(codmat)..."
context menu "Sterge" → deleteMappingConfirm → modal confirm → fetch
/api/mappings/delete?sku=&codmat=. Exercises BOTH the slash (routing, query
param fix) and the quote (attribute escaping, commit a530ebf) end-to-end.
Needs a real Oracle-backed app (seed inserts into ARTICOLE_TERTI with a real
CODMAT from NOM_ARTICOLE). When run against the dummy-Oracle subprocess in the
plain e2e layer, seeding fails and the test skips — it only asserts in the
./test.sh full (or live :5003) layer.
"""
import pytest
from playwright.sync_api import Page, expect
pytestmark = pytest.mark.e2e
SLASH_SKU = 'RCR1/4"'
def _real_codmat(page: Page, app_url: str):
"""Fetch a real CODMAT from the Oracle nomenclator, or None if Oracle is down."""
for term in ["01", "PH", "CA"]:
resp = page.request.get(f"{app_url}/api/articles/search?q={term}")
if resp.ok:
results = resp.json().get("results", [])
if results:
return results[0]["codmat"]
return None
def test_delete_mapping_with_slash_and_quote(page: Page, app_url: str):
codmat = _real_codmat(page, app_url)
if not codmat:
pytest.skip("Oracle unavailable (no real CODMAT) — slash-delete e2e needs the full layer")
# Seed: create the slash+quote mapping. Restore if a prior run soft-deleted it.
resp = page.request.post(f"{app_url}/api/mappings",
data={"sku": SLASH_SKU, "codmat": codmat, "cantitate_roa": 1})
body = resp.json() if resp.status in (200, 409) else {}
if not body.get("success"):
if body.get("can_restore"):
r = page.request.post(
f"{app_url}/api/mappings/restore",
params={"sku": SLASH_SKU, "codmat": codmat})
assert r.ok and r.json().get("success"), f"restore seed failed: {r.status} {r.text()}"
elif resp.status == 503:
pytest.skip("Oracle unavailable (503 on create) — slash-delete e2e needs the full layer")
else:
pytest.fail(f"could not seed mapping: {resp.status} {resp.text()}")
# Collect any 404 on the mappings API during the UI flow.
not_found = []
page.on("response", lambda r: not_found.append(r.url)
if r.status == 404 and "/api/mappings/" in r.url else None)
page.goto(f"{app_url}/mappings")
page.wait_for_load_state("networkidle")
# Search for the slash+quote SKU.
page.fill("#searchInput", SLASH_SKU)
page.wait_for_timeout(600) # debounce + reload
# The row's context-menu trigger carries the codmat as a data attribute.
trigger = page.locator(f'.context-menu-trigger[data-sku="{SLASH_SKU}"]').first
expect(trigger).to_be_visible()
trigger.click()
# Context menu → "Sterge" → confirm modal.
page.locator(".context-menu-item", has_text="Sterge").click()
confirm = page.locator("#confirmDeleteBtn")
expect(confirm).to_be_visible()
confirm.click()
page.wait_for_timeout(600) # delete fetch + reload
# Row is gone and no 404 was hit on the mappings API.
expect(page.locator(f'.context-menu-trigger[data-sku="{SLASH_SKU}"]')).to_have_count(0)
assert not not_found, f"mappings API returned 404 during delete flow: {not_found}"

View File

@@ -112,3 +112,25 @@ def test_route(client, path, expected_codes, is_oracle_route):
f"GET {path} returned {resp.status_code}, expected one of {expected_codes}. "
f"Body: {resp.text[:300]}"
)
def test_mappings_slash_in_keys_routes_ok(client):
"""Regression: slash in BOTH sku and codmat used to 404 (path-param routes,
ASGI decoded %2F before routing). Now sku/codmat are query params, so the
route resolves regardless of slashes/quotes. Sentinel keys with a slash can
match no real row, so even if Oracle were up the UPDATE/WHERE is a no-op —
here pool is None so the handler returns 200 success:False without mutation.
Asserting != 404 proves the route resolved."""
sku, codmat = '__TEST_SKU/X"', '__TEST_CODMAT/Y'
calls = [
client.delete("/api/mappings/delete", params={"sku": sku, "codmat": codmat}),
client.put("/api/mappings/update", params={"sku": sku, "codmat": codmat},
json={"cantitate_roa": 1}),
client.put("/api/mappings/edit", params={"sku": sku, "codmat": codmat},
json={"new_sku": sku, "new_codmat": codmat, "cantitate_roa": 1}),
client.post("/api/mappings/restore", params={"sku": sku, "codmat": codmat}),
]
for resp in calls:
assert resp.status_code != 404, (
f"route did not resolve: {resp.status_code} {resp.url}"
)

View File

@@ -126,16 +126,17 @@ def test_mappings_list_after_create(client, real_codmat, test_sku):
def test_mappings_update(client, real_codmat, test_sku):
resp = client.put(f"/api/mappings/{test_sku}/{real_codmat}", json={
"cantitate_roa": 3.0,
})
resp = client.put("/api/mappings/update",
params={"sku": test_sku, "codmat": real_codmat},
json={"cantitate_roa": 3.0})
assert resp.status_code == 200
body = resp.json()
assert body.get("success") is True, f"update returned: {body}"
def test_mappings_delete(client, real_codmat, test_sku):
resp = client.delete(f"/api/mappings/{test_sku}/{real_codmat}")
resp = client.delete("/api/mappings/delete",
params={"sku": test_sku, "codmat": real_codmat})
assert resp.status_code == 200
body = resp.json()
assert body.get("success") is True, f"delete returned: {body}"

View File

@@ -32,6 +32,9 @@ client = TestClient(app)
@pytest.fixture(autouse=True)
async def _reset():
database.init_sqlite()
# Simulate Oracle up for health tests (no real pool in unit env).
_orig_pool = database.pool
database.pool = object()
db = await sqlite_service.get_sqlite()
try:
await db.execute("DELETE FROM sync_phase_failures")
@@ -40,6 +43,7 @@ async def _reset():
finally:
await db.close()
yield
database.pool = _orig_pool
async def _make_run(run_id: str, status: str = "completed", offset: int = 0,
@@ -108,3 +112,12 @@ async def test_health_one_phase_failure_still_warning_not_healthy():
# 1 recent phase failure → is_healthy stays True (<=1 tolerance); healthy
assert data["is_healthy"] is True
assert data["recent_phase_failures"]["invoice_check"] == 1
async def test_health_oracle_down_not_healthy():
await _make_run("ok-oracle", status="completed")
database.pool = None # simulate Oracle pool not initialized
r = client.get("/api/sync/health")
data = r.json()
assert data["oracle_ready"] is False
assert data["is_healthy"] is False

View File

@@ -35,9 +35,9 @@ param(
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# Helpers
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
function Write-Step { param([string]$msg) Write-Host "`n==> $msg" -ForegroundColor Cyan }
function Write-OK { param([string]$msg) Write-Host " [OK] $msg" -ForegroundColor Green }
function Write-Warn { param([string]$msg) Write-Host " [WARN] $msg" -ForegroundColor Yellow }
@@ -46,9 +46,9 @@ function Write-Info { param([string]$msg) Write-Host " $msg" -Foregroun
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 1. Citire token Gitea
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Citire token Gitea"
$TokenFile = Join-Path $ScriptDir ".gittoken"
@@ -71,9 +71,9 @@ $RepoUrl = if ($GitToken) {
"https://gitea.romfast.ro/romfast/gomag-vending.git"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 2. Git clone / pull
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Git clone / pull"
# Verifica git instalat
@@ -106,9 +106,9 @@ if (Test-Path (Join-Path $RepoPath ".git")) {
Write-OK "git clone OK"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 3. Verificare Python
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Verificare Python"
$PythonCmd = $null
@@ -135,9 +135,9 @@ if (-not $PythonCmd) {
exit 1
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 4. Creare venv si instalare dependinte
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Virtual environment + dependinte"
$VenvDir = Join-Path $RepoPath "venv"
@@ -170,9 +170,9 @@ if ($needInstall) {
Write-OK "Dependinte deja up-to-date"
}
# ─────────────────────────────────────────────────────────────────────────────
# 5. Detectare Oracle Home sugestie INSTANTCLIENTPATH
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 5. Detectare Oracle Home -> sugestie INSTANTCLIENTPATH
# -----------------------------------------------------------------------------
Write-Step "Detectare Oracle"
$OracleHome = $env:ORACLE_HOME
@@ -207,9 +207,9 @@ if ($OracleHome -and (Test-Path $OracleHome)) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 6. Creare .env din template daca lipseste
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Fisier configurare api\.env"
$EnvFile = Join-Path $RepoPath "api\.env"
@@ -241,9 +241,9 @@ if (-not (Test-Path $EnvFile)) {
Write-OK "api\.env exista deja"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 7. Creare directoare necesare
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Directoare date"
foreach ($dir in @("data", "output", "logs")) {
@@ -256,9 +256,9 @@ foreach ($dir in @("data", "output", "logs")) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 8. Generare start.bat
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Generare start.bat"
$StartBat = Join-Path $RepoPath "start.bat"
@@ -298,13 +298,13 @@ echo Starting GoMag Import Manager on http://0.0.0.0:$Port (prefix /gomag)
Set-Content -Path $StartBat -Value $StartBatContent -Encoding UTF8
Write-OK "start.bat generat: $StartBat"
# ─────────────────────────────────────────────────────────────────────────────
# 9. IIS Verificare ARR + URL Rewrite
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 9. IIS - Verificare ARR + URL Rewrite
# -----------------------------------------------------------------------------
Write-Step "Verificare module IIS"
if ($SkipIIS) {
Write-Warn "SkipIIS activ configurare IIS sarita"
Write-Warn "SkipIIS activ - configurare IIS sarita"
} else {
$ArrPath = "$env:SystemRoot\System32\inetsrv\arr.dll"
$UrlRewritePath = "$env:SystemRoot\System32\inetsrv\rewrite.dll"
@@ -328,9 +328,9 @@ if ($SkipIIS) {
Write-Info "Sau: winget install Microsoft.URLRewrite"
}
# ─────────────────────────────────────────────────────────────────────────
# 10. Configurare IIS copiere web.config
# ─────────────────────────────────────────────────────────────────────────
# -------------------------------------------------------------------------
# 10. Configurare IIS - copiere web.config
# -------------------------------------------------------------------------
if ($ArrOk -and $UrlRwOk) {
Write-Step "Configurare IIS reverse proxy"
@@ -354,7 +354,7 @@ if ($SkipIIS) {
}
} catch {
Write-Warn "Nu am putut activa ARR proxy automat: $($_.Exception.Message)"
Write-Info "Activeaza manual din IIS Manager server root Application Request Routing Cache Enable Proxy"
Write-Info "Activeaza manual din IIS Manager -> server root -> Application Request Routing Cache -> Enable Proxy"
}
# Determina wwwroot site-ului IIS
@@ -406,13 +406,13 @@ if ($SkipIIS) {
}
}
} else {
Write-Warn "IIS nu e configurat complet instaleaza ARR si URL Rewrite, apoi ruleaza deploy.ps1 din nou"
Write-Warn "IIS nu e configurat complet - instaleaza ARR si URL Rewrite, apoi ruleaza deploy.ps1 din nou"
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 11. Serviciu Windows (NSSM sau Task Scheduler)
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Serviciu Windows"
$ServiceName = "GoMagVending"
@@ -431,25 +431,38 @@ if ($NssmExe) {
$existingService = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($existingService) {
Write-Info "Serviciu existent, restarteaza..."
& $NssmExe restart $ServiceName
Write-OK "Serviciu $ServiceName restartat"
} else {
if (-not $existingService) {
Write-Info "Instalez serviciu $ServiceName cu NSSM..."
& $NssmExe install $ServiceName (Join-Path $RepoPath "start.bat")
& $NssmExe set $ServiceName AppDirectory $RepoPath
& $NssmExe set $ServiceName DisplayName "GoMag Vending Import Manager"
& $NssmExe set $ServiceName Description "Import comenzi web GoMag -> ROA Oracle"
& $NssmExe set $ServiceName Start SERVICE_AUTO_START
} else {
Write-Info "Serviciu existent, il opresc pentru mentenanta..."
& $NssmExe stop $ServiceName 2>$null
Start-Sleep -Seconds 3
}
# Mentenanta DB + log-uri cu serviciul oprit: prune istoric (7z) + VACUUM
# (reclaim disc) + cleanup log-uri (7z). Non-fatal daca esueaza.
Write-Info "Mentenanta DB/log-uri (prune + VACUUM + cleanup)..."
try {
& $VenvPy (Join-Path $RepoPath "scripts\db_maintenance.py") --history-days 7 --log-days 7
} catch {
Write-Warn "Mentenanta DB a esuat (continui): $_"
}
# (Re)aplica config log/rotatie de fiecare data - idempotent, astfel rotatia
# ajunge si pe serviciile instalate inainte ca aceste setari sa existe.
& $NssmExe set $ServiceName AppStdout (Join-Path $RepoPath "logs\service_stdout.log")
& $NssmExe set $ServiceName AppStderr (Join-Path $RepoPath "logs\service_stderr.log")
& $NssmExe set $ServiceName AppRotateFiles 1
& $NssmExe set $ServiceName AppRotateOnline 1
& $NssmExe set $ServiceName AppRotateBytes 10485760
& $NssmExe start $ServiceName
Write-OK "Serviciu $ServiceName instalat si pornit"
}
Write-OK "Serviciu $ServiceName pornit (mentenanta + rotatie aplicate)"
} else {
# Fallback: Task Scheduler
@@ -498,13 +511,13 @@ if ($NssmExe) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# Sumar final
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Host ""
Write-Host "══════════════════════════════════════════════════════" -ForegroundColor Cyan
Write-Host " GoMag Vending Deploy Sumar" -ForegroundColor Cyan
Write-Host "══════════════════════════════════════════════════════" -ForegroundColor Cyan
Write-Host "======================================================" -ForegroundColor Cyan
Write-Host " GoMag Vending Deploy - Sumar" -ForegroundColor Cyan
Write-Host "======================================================" -ForegroundColor Cyan
Write-Host ""
Write-Host " Repo: $RepoPath" -ForegroundColor White
Write-Host " FastAPI: http://localhost:$Port/gomag" -ForegroundColor White
@@ -512,13 +525,13 @@ Write-Host " start.bat generat" -ForegroundColor White
Write-Host ""
if (-not (Test-Path $EnvFile)) {
Write-Host " [!] api\.env lipseste configureaza inainte de start!" -ForegroundColor Red
Write-Host " [!] api\.env lipseste - configureaza inainte de start!" -ForegroundColor Red
} else {
Write-Host " api\.env: OK" -ForegroundColor Green
# Verifica daca mai are valori placeholder
$envContent = Get-Content $EnvFile -Raw
if ($envContent -match "your_api_key_here|USER_ORACLE|parola_oracle|TNS_ALIAS") {
Write-Host " [!] api\.env contine valori placeholder editeaza!" -ForegroundColor Yellow
Write-Host " [!] api\.env contine valori placeholder - editeaza!" -ForegroundColor Yellow
}
}

View File

@@ -0,0 +1,379 @@
# Căutarea și selecția clientului după codul fiscal (cu verificare ANAF)
> Document de referință pentru replicarea logicii într-un alt proiect.
> Descrie **unde** se caută, **cum** se normalizează codul fiscal, **ce** se verifică pe ANAF
> și **procedurile** (Python + Oracle PL/SQL) implicate, end-to-end.
Sistemul curent: GoMag (web) → ROA Oracle. Partenerii sunt căutați/creați în tabela Oracle
`nom_parteneri`, iar codul fiscal este validat și corectat folosind webserviciul ANAF.
---
## 0. Fișiere sursă (în acest proiect)
| Componentă | Fișier | Funcții cheie |
|---|---|---|
| Validare/normalizare CUI + apel ANAF | `api/app/services/anaf_service.py` | `strip_ro_prefix`, `validate_cui`, `validate_cui_checksum`, `sanitize_cui`, `check_vat_status_batch`, `_call_anaf_api`, `determine_correct_cod_fiscal`, `normalize_company_name` |
| Extragere date partener din comandă | `api/app/services/import_service.py` | `determine_partner_data` |
| Orchestrare sync + gate CUI + override | `api/app/services/sync_service.py` | `evaluate_cui_gate`, secțiunea Step 4/5 (ANAF batch, anaf_strict, overrides) |
| Căutare/creare partener (Oracle) | `api/database-scripts/05_pack_import_parteneri.pck` | `cauta_partener_dupa_cod_fiscal`, `cauta_sau_creeaza_partener`, `cauta_partener_dupa_denumire` |
| Teste | `api/tests/test_cui_validation.py`, `test_partner_cui_lookup.py`, `test_partner_anaf_override.py`, `test_sync_cui_gate.py` | — |
---
## 1. Privire de ansamblu (flux complet)
```
Comandă web (JSON)
determine_partner_data() # PJ vs PF, extrage cod_fiscal/denumire/registru
▼ (doar PJ romanesc cu cod_fiscal)
sanitize_cui() # scoate RO, corectează typo OCR, validează format+checksum
check_vat_status_batch() → ANAF # POST batch la webserviciul ANAF (platitor TVA?)
│ răspuns: {scpTVA, denumire_anaf, checked_at} (cache 7 zile in SQLite)
evaluate_cui_gate() # BLOCHEAZĂ comanda dacă: format invalid / checksum gresit / ANAF notFound
│ (ANAF down → tolerează, lasă să treacă)
determine_correct_cod_fiscal() # aplică prefix RO dacă e plătitor TVA
anaf_strict = 1 # marchează că avem date ANAF → căutare STRICTĂ în Oracle
denumire_override = denumire_anaf # la CREARE partener nou folosește numele oficial ANAF
PACK_IMPORT_PARTENERI.cauta_sau_creeaza_partener(cod_fiscal, denumire, registru, is_pj, anaf_strict, OUT id_part)
├─ STEP 1: cauta_partener_dupa_cod_fiscal() # prioritate 1
├─ STEP 2: cauta_partener_dupa_denumire() # prioritate 2 (sărit în mod strict)
├─ STEP 2b: permutări nume (doar PF, non-strict)
└─ STEP 3: creare partener nou (pack_def.adauga_partener)
```
Principiu central: **codul fiscal are prioritate** în identificarea partenerului; numele e doar
fallback. ANAF servește la (a) a decide forma corectă a CUI (cu/fără `RO`) și (b) a bloca CUI-uri
inexistente/invalide înainte de a polua baza cu parteneri greșiți.
---
## 2. Normalizarea codului fiscal (Python)
Fișier: `api/app/services/anaf_service.py`.
### 2.1 `strip_ro_prefix(cod_fiscal) -> str`
Aduce CUI-ul la forma „bare” (doar cifre):
1. `strip().upper()`
2. elimină prefixul `RO` cu spațiu opțional: `re.sub(r'^RO\s*', '', cleaned)`
3. corectează confuzii OCR frecvente: `O→0`, `I→1`, `L→1` (via `str.maketrans('OIL', '011')`)
```python
def strip_ro_prefix(cod_fiscal: str) -> str:
if not cod_fiscal:
return ""
cleaned = cod_fiscal.strip().upper()
cleaned = re.sub(r'^RO\s*', '', cleaned)
cleaned = cleaned.translate(str.maketrans('OIL', '011'))
return cleaned
```
În plus, la extragerea din comandă (`import_service.determine_partner_data`) se colapsează
**tot** spațiul intern: `cod_fiscal = re.sub(r'\s+', '', raw_cf)``"RO 34963277"` devine `"RO34963277"`.
### 2.2 `validate_cui(bare_cui) -> bool`
Format: doar cifre, lungime 210.
### 2.3 `validate_cui_checksum(bare_cui) -> bool` — algoritmul oficial românesc
Cheia de control: `[7, 5, 3, 2, 1, 7, 5, 3, 2]` (9 ponderi).
1. ultima cifră = cifra de control; corpul = restul cifrelor
2. corpul se aliniază la dreapta pe 9 poziții (padding cu 0 la stânga)
3. `total = Σ(cifra_i * pondere_i)`
4. `result = (total * 10) % 11`; dacă `result == 10``result = 0`
5. valid dacă `result == cifra_de_control`
```python
_CUI_KEY = [7, 5, 3, 2, 1, 7, 5, 3, 2]
def validate_cui_checksum(bare_cui: str) -> bool:
if not validate_cui(bare_cui):
return False
digits = [int(d) for d in bare_cui]
check_digit = digits[-1]
body = digits[:-1]
padded = [0] * (9 - len(body)) + body
total = sum(d * k for d, k in zip(padded, _CUI_KEY))
result = (total * 10) % 11
if result == 10:
result = 0
return result == check_digit
```
### 2.4 `sanitize_cui(raw_cf) -> (bare_cui, warning|None)`
Combină normalizarea + validarea și produce avertisment, fără a arunca excepție:
- strip RO + typo fix
- dacă format **și** checksum OK → `(bare, None)`
- dacă format OK dar checksum greșit → `(bare, "CUI ... nu trece verificarea cifrei de control")`
- dacă nici format → `(bare, "CUI ... conține caractere invalide...")`
### 2.5 `determine_correct_cod_fiscal(bare_cui, is_vat_payer) -> str`
Decide forma corectă pe baza statusului TVA ANAF:
- `is_vat_payer is True``"RO" + bare`
- `False` sau `None``bare` (conservator)
### 2.6 `normalize_company_name(name)` (auxiliar, comparare nume firme)
Uppercase, scoate diacritice, elimină forme juridice (`SRL/SA/SC/SNC/PFA/II/Întreprindere Individuală`),
elimină punctuația, colapsează spațiile. Util pentru comparații nume, nu pentru lookup-ul principal.
---
## 3. Ce se verifică pe ANAF și cum
Fișier: `api/app/services/anaf_service.py`.
### 3.1 Endpoint
```
POST https://webservicesp.anaf.ro/api/PlatitorTvaRest/v9/tva
Content-Type: application/json
Body: [{"cui": <int>, "data": "YYYY-MM-DD"}, ...] # batch, max 500 / cerere
```
### 3.2 Câmpuri citite din răspuns
Pentru fiecare CUI din lista `found`:
- `inregistrare_scop_Tva.scpTVA`**plătitor TVA?** (bool)
- `date_generale.denumire`**denumire oficială ANAF** (`denumire_anaf`)
- `date_generale.cui` → CUI (cheia rezultatului)
Plus `checked_at` (timestamp local), folosit pentru validitatea cache-ului.
CUI-urile din lista `notFound` (ANAF le întoarce ca **întregi simpli**, nu obiecte) primesc
`{scpTVA: None, denumire_anaf: ""}`.
### 3.3 Contractul de date ANAF (foarte important)
Restul sistemului interpretează rezultatul ANAF strict astfel:
| Rezultat | Semnificație | Acțiune |
|---|---|---|
| `None` (lipsă din dict / batch eșuat) | ANAF down / eroare tranzitorie | **tolerează** (lasă să treacă) |
| `{scpTVA: None, denumire_anaf: ""}` | ANAF notFound explicit | **blochează** comanda |
| `{scpTVA: bool, denumire_anaf: str}` | ANAF found | **trece** |
> Dacă schimbi această semantică în clientul ANAF, trebuie actualizat și `evaluate_cui_gate`.
### 3.4 Robustețe apel (`_call_anaf_api`)
- timeout 10s
- `429` → 1 retry după 10s, apoi renunță (`{}`)
- `5xx` → 1 retry după 3s, apoi renunță (`{}`)
- `4xx` → fail-fast, **fără** retry (`{}`)
- timeout / excepție → 1 retry după 3s
- „renunță” înseamnă întoarce `{}` ⇒ tratat ca *transient* ⇒ comanda **nu** e blocată
### 3.5 Cache & batching (orchestrare în `sync_service`)
- se adună CUI-urile firmelor **RO** din comenzile importabile (`sanitize_cui` + `validate_cui`)
- se verifică cache-ul SQLite (`anaf_cache`, valabilitate 7 zile); doar CUI-urile necacheate intră în batch
- pre-populare: la fiecare sync se reîmprospătează CUI-urile din ultimele 3 luni cu cache expirat
- rezultatele batch se scriu în cache (`bulk_populate_anaf_cache`)
---
## 4. „Gate”-ul CUI — când se blochează comanda
Fișier: `api/app/services/sync_service.py`, funcția `evaluate_cui_gate`.
```python
def evaluate_cui_gate(is_ro_company, company_code_raw, bare_cui, anaf_data) -> str | None:
if not is_ro_company or not company_code_raw:
return None # PF sau firmă non-RO → trece
if not anaf_service.validate_cui(bare_cui):
return f"CUI invalid (format): {company_code_raw!r}"
if not anaf_service.validate_cui_checksum(bare_cui):
return f"CUI invalid (cifra de control): {bare_cui}"
if (anaf_data is not None
and anaf_data.get("scpTVA") is None
and not (anaf_data.get("denumire_anaf") or "").strip()):
return f"CUI {company_code_raw!r} ... nu exista in registrul ANAF ..."
return None # altfel trece
```
Reguli rezumate:
- **PF** sau **firmă non-RO** → trece necondiționat (fără verificare ANAF/CUI)
- **PJ RO**: blochează la format invalid, checksum greșit, sau ANAF notFound explicit
- **ANAF down** (`anaf_data is None`) → trece (toleranță, ca să nu blocăm tot importul când ANAF pică)
La blocare se scrie o comandă cu status `ERROR` (nu se creează partener). Există un prag
de siguranță: după >10 erori, importul se oprește.
---
## 5. Determinarea modului de căutare și a override-urilor (Python)
Fișier: `api/app/services/sync_service.py` (în bucla de import, Step 5).
Pentru fiecare comandă PJ RO cu date ANAF valide (`scpTVA is not None`):
1. **`cod_fiscal_override`** = `determine_correct_cod_fiscal(bare_cui, scpTVA)` — corectează forma `RO`/bare
2. **`anaf_strict = 1`** — semnal pentru Oracle că avem certitudine ANAF ⇒ căutare strictă
3. **`denumire_override`** = `denumire_anaf` (uppercase, doar dacă non-gol) — folosit **doar la creare** partener nou
Aceste valori se transmit către `import_single_order(...)`, care apoi cheamă procedura Oracle.
`anaf_strict` rămâne `None` (fallback non-strict) când: firmă non-RO, fără date ANAF, sau ANAF down.
---
## 6. Căutarea/selecția în Oracle
Fișier: `api/database-scripts/05_pack_import_parteneri.pck`.
### 6.1 `cauta_partener_dupa_cod_fiscal(p_cod_fiscal, p_strict_search) RETURN NUMBER`
Tabela: `nom_parteneri`, coloana `cod_fiscal`. Întotdeauna filtrează `NVL(sters,0)=0`
(exclude șterse) și preferă activii (`NVL(inactiv,0) ASC`), apoi `id_part DESC` (cel mai nou).
Pași comuni:
1. `RETURN NULL` dacă `p_cod_fiscal` e NULL sau mai scurt de `C_MIN_COD_FISCAL`
2. `v_cod_fiscal_curat := UPPER(TRIM(...))`
3. extrage `v_bare_cui` (fără RO) cu `REGEXP_LIKE(..., '^RO\s*\d')` + `REGEXP_REPLACE`
4. `v_ro_cui := 'RO' || v_bare_cui`
**Mod STRICT (`p_strict_search = 1`)** — diferențiază plătitor vs neplătitor TVA, **fără cross-match**
(sunt entități fiscale distincte):
- input cu prefix RO (`^RO\s*\d`, plătitor TVA) → caută **doar** `RO<bare>` și `RO <bare>` (cu spațiu)
- input bare (neplătitor) → caută **doar** `<bare>`
**Mod NON-STRICT (`p_strict_search IS NULL`)** — anti-duplicare, caută toate cele 3 forme
(`bare`, `RO<bare>`, `RO <bare>`), cu prioritate pe forma exactă a input-ului.
```sql
FUNCTION cauta_partener_dupa_cod_fiscal(p_cod_fiscal IN VARCHAR2,
p_strict_search IN NUMBER DEFAULT NULL)
RETURN NUMBER IS
v_id_part NUMBER; v_cod_fiscal_curat VARCHAR2(50);
v_bare_cui VARCHAR2(50); v_ro_cui VARCHAR2(52);
BEGIN
IF p_cod_fiscal IS NULL OR LENGTH(TRIM(p_cod_fiscal)) < C_MIN_COD_FISCAL THEN
RETURN NULL;
END IF;
v_cod_fiscal_curat := UPPER(TRIM(p_cod_fiscal));
IF REGEXP_LIKE(v_cod_fiscal_curat, '^RO\s*\d') THEN
v_bare_cui := TRIM(REGEXP_REPLACE(v_cod_fiscal_curat, '^RO\s*', ''));
ELSE
v_bare_cui := v_cod_fiscal_curat;
END IF;
v_ro_cui := 'RO' || v_bare_cui;
BEGIN
IF p_strict_search = 1 THEN
IF REGEXP_LIKE(v_cod_fiscal_curat, '^RO\s*\d') THEN
SELECT id_part INTO v_id_part FROM (
SELECT id_part FROM nom_parteneri
WHERE UPPER(TRIM(cod_fiscal)) IN (v_ro_cui, 'RO ' || v_bare_cui)
AND NVL(sters, 0) = 0
ORDER BY NVL(inactiv, 0) ASC, id_part DESC
) WHERE ROWNUM = 1;
ELSE
SELECT id_part INTO v_id_part FROM (
SELECT id_part FROM nom_parteneri
WHERE UPPER(TRIM(cod_fiscal)) = v_bare_cui
AND NVL(sters, 0) = 0
ORDER BY NVL(inactiv, 0) ASC, id_part DESC
) WHERE ROWNUM = 1;
END IF;
ELSE
SELECT id_part INTO v_id_part FROM (
SELECT id_part FROM nom_parteneri
WHERE UPPER(TRIM(cod_fiscal)) IN (v_bare_cui, v_ro_cui, 'RO ' || v_bare_cui)
AND NVL(sters, 0) = 0
ORDER BY NVL(inactiv, 0) ASC,
CASE WHEN UPPER(TRIM(cod_fiscal)) = v_cod_fiscal_curat THEN 0 ELSE 1 END ASC,
id_part DESC
) WHERE ROWNUM = 1;
END IF;
RETURN v_id_part;
EXCEPTION WHEN NO_DATA_FOUND THEN RETURN NULL;
END;
EXCEPTION WHEN OTHERS THEN
pINFO('ERROR in cauta_partener_dupa_cod_fiscal: ' || SQLERRM, 'IMPORT_PARTENERI');
RAISE;
END;
```
### 6.2 `cauta_sau_creeaza_partener(...)` — orchestrarea selecției/creării
Semnătură:
```sql
PROCEDURE cauta_sau_creeaza_partener(
p_cod_fiscal IN VARCHAR2,
p_denumire IN VARCHAR2,
p_registru IN VARCHAR2,
p_is_persoana_juridica IN NUMBER DEFAULT NULL, -- 1=PJ, 0=PF, NULL=auto-detect CNP
p_strict_search IN NUMBER DEFAULT NULL, -- 1=avem date ANAF
p_id_partener OUT NUMBER) -- >0 ok, -1 eroare
```
Ordinea priorităților:
- **STEP 1 — după cod fiscal** (prioritate maximă): `cauta_partener_dupa_cod_fiscal(cod, strict)`.
Dacă găsește → return.
- **STEP 2 — după denumire exactă**: `cauta_partener_dupa_denumire` (match uppercase exact).
**Sărit** când `p_strict_search` e setat (în mod ANAF strict vrem partener nou cu CUI corect,
nu unul vechi găsit doar după nume).
- **STEP 2b — permutări nume** (doar PF, doar non-strict): rezolvă inversarea nume/prenume din web.
2 cuvinte → încearcă `W2 W1`; 3 cuvinte → încearcă celelalte 5 permutări.
- **STEP 3 — creare partener nou** (`pack_def.adauga_partener`):
- tip: parametru explicit `p_is_persoana_juridica` are prioritate; altfel auto-detect prin CNP (13 cifre)
- PF: `separa_nume_prenume`, `tnTip_persoana => 2`
- PJ: `tnTip_persoana => 1`
- denumirea se trece prin `strip_diacritics` înainte de stocare
- return `-1` la eșec
### 6.3 `cauta_partener_dupa_denumire(p_denumire)`
Match exact pe denumire normalizată (`UPPER(TRIM(...))`). Folosit ca fallback secundar și pentru permutările PF.
---
## 7. PJ vs PF: cum se decide tipul și ce date se folosesc
Fișier: `api/app/services/import_service.py`, `determine_partner_data`.
- **PJ (persoană juridică)** — când comanda are flag `billing.is_company`:
- `denumire` = numele firmei (curățat, uppercase); fallback la numele persoanei din billing dacă lipsește
- `cod_fiscal` = codul firmei cu spațiul colapsat (`re.sub(r'\s+','',...)`)
- `registru` = registrul comerțului
- `is_pj = 1`
- **PF (persoană fizică)** — altfel:
- `denumire` = numele din **shipping** (fallback billing), cuvinte **sortate alfabetic**
(`" ".join(sorted(...))`) — stabilizează ordinea pentru match
- `cod_fiscal = None`, `registru = None`, `is_pj = 0`
La creare partener **nou PJ** se preferă numele oficial ANAF (`denumire_override`) în locul numelui
din web (care poate avea typo-uri). Partenerii **existenți nu sunt modificați**.
---
## 8. Exemple (input → comportament)
| Input cod_fiscal | scpTVA (ANAF) | anaf_strict | Caută în `nom_parteneri` |
|---|---|---|---|
| `"RO 34963277"` | True | 1 | `RO34963277`, `RO 34963277` (doar formele RO) |
| `"34963277"` (plătitor) | True | 1 | corectat → `RO34963277` (override), apoi formele RO |
| `"34963277"` (neplătitor) | False | 1 | doar `34963277` (bare) |
| `"34963277"` | — (ANAF down) | NULL | toate 3 formele (anti-dedup) |
| `"RO123"` strict | — | 1 | nu face match pe `123` bare (entități distincte) |
| CUI checksum greșit | — | — | **BLOCAT** la gate (status ERROR) |
| CUI inexistent ANAF | None+gol | — | **BLOCAT** la gate |
| PF (fără cod) | — | — | căutare după nume (+permutări), apoi creare PF |
---
## 9. Checklist de replicare în alt proiect
1. **Normalizare CUI**: strip `RO` (cu spațiu opțional), fix typo OCR (`O→0,I→1,L→1`), colapsare spații.
2. **Validare**: format (210 cifre) + checksum românesc (cheia `7,5,3,2,1,7,5,3,2`, `(Σ*10)%11`, 10→0).
3. **Client ANAF**: POST batch (≤500) la `v9/tva`, citește `scpTVA` + `denumire`; respectă contractul
tri-valent (None=down→tolerează, notFound→blochează, found→trece); retry doar pe 429/5xx/timeout.
4. **Cache**: stochează rezultatele ANAF (ex. 7 zile) ca să nu reinteroghezi la fiecare rulare.
5. **Gate**: blochează doar PJ RO la format/checksum/notFound; PF și ANAF-down trec.
6. **Forma corectă CUI**: aplică `RO` doar dacă `scpTVA = True`.
7. **Căutare DB**: prioritate cod fiscal → nume exact → (PF) permutări nume → creare.
În mod strict (date ANAF), separă plătitor (`RO...`) de neplătitor (`bare`) **fără cross-match**;
în mod non-strict, caută toate formele cu prioritate pe forma exactă.
8. **Creare partener nou**: nume oficial ANAF pentru PJ; nu atinge partenerii existenți.
9. **Excluderi**: ignoră înregistrările șterse; preferă active + cea mai nouă la ambiguitate.

View File

@@ -0,0 +1,133 @@
# Reconciliere facturi manuale ROA ↔ comenzi GoMag
**Script:** [`scripts/relink_manual_invoices.py`](../scripts/relink_manual_invoices.py)
**Launcher Windows:** [`scripts/relink_manual_invoices.bat`](../scripts/relink_manual_invoices.bat)
## Cand se foloseste
Cand Oracle pool / sync-ul **a fost cazut** (ex. dupa o pana de curent la VENDING)
si operatorul a emis **facturi manual direct in ROA** in acel interval. Dupa ce
aplicatia revine, importeaza aceleasi comenzi web in `COMENZI`, **dar** factura
manuala nu se leaga de comanda → comanda ramane **nefacturata** in ROA si in
dashboard, desi factura exista.
Semnal: in dashboard apar comenzi „nefacturate" pentru clienti care au fost de
fapt facturati, iar in `VANZARI` exista facturi cu `ID_COMANDA IS NULL`.
## Cauza tehnica
| Flux | `VANZARI.ID_COMANDA` | `COMENZI.COMANDA_EXTERNA` |
|------|----------------------|----------------------------|
| Normal (app) | setat (leaga factura de comanda) | nr comanda GoMag |
| Manual (operator, in downtime) | **NULL** | — |
Dashboard-ul / cache-ul SQLite marcheaza o comanda „Facturat" doar daca exista
`VANZARI ... WHERE id_comanda = <comanda> AND sters = 0`
(vezi `invoice_service.check_invoices_for_orders`). Factura manuala cu
`ID_COMANDA NULL` nu e niciodata gasita → comanda apare nefacturata.
`ID_FACT` (documentul fiscal) si `COMENZI.COMANDA_EXTERNA` sunt deja completate;
**singura piesa lipsa e legatura `VANZARI → COMANDA`.**
## ⚠️ Facturi de depozit (walk-in) — NU se ating
Operatorul emite zilnic si **facturi manuale legitime din depozit** (~20+/zi),
fara nicio comanda online in spate. Acestea au tot `ID_COMANDA NULL`, dar **nu**
trebuie legate de nimic. De aceea matching-ul e **conservator**: orice nu e o
potrivire 1:1 sigura e raportat, niciodata legat automat.
## Cum potriveste
Pentru fiecare factura orfana (`VANZARI.ID_COMANDA NULL`, `sters=0`, in fereastra
`--days`) cauta o comanda GoMag nefacturata (in SQLite, cu `id_comanda` setat) cu:
1. **Total identic** (`TOTAL_CU_TVA``order_total`, toleranta 0.01 lei), apoi
2. **acelasi partener** (`ID_PART` = `id_partener`) **SAU**
3. **nume potrivit** (token-overlap, tolereaza SRL/SC si ordinea cuvintelor) —
acopera cazul in care operatorul a creat un **partener duplicat** la facturare.
Verifica si ca acea comanda e activa in ROA (`sters=0`) si **nu** are deja factura
(anti-dubla-facturare).
### Clasificare (in raport)
| Eticheta | Ce inseamna | Actiune |
|----------|-------------|---------|
| `LINK` | potrivire 1:1 sigura | leaga (la `--apply`) |
| `SKIP_NOMATCH` | nicio comanda online cu acel total | **factura de depozit — lasata neatinsa** |
| `SKIP_AMBIGUOUS` | mai multe comenzi plauzibile, sau total potrivit dar partener+nume diferit | raportat pentru verificare manuala |
| `SKIP_ALREADY` | comanda nu mai e activa / are deja factura | sarit |
La aplicare: `UPDATE VANZARI SET ID_COMANDA = <comanda>` + populeaza
`orders.factura_*` in SQLite, exact ca aplicatia (`update_order_invoice`).
### ⚠ Verificare reziduu de linie (legatura header nu e mereu suficienta)
Aplicatia / dashboard-ul marcheaza o comanda „Facturat" doar dupa **legatura header**
(`VANZARI.ID_COMANDA`). **ROA insa verifica la nivel de linie**: in
`PACK_FACTURARE.cursor_comanda`, cantitatea facturata se potriveste cu comanda pe
**`ID_ARTICOL` + `PRET` exact**, iar o linie e „de facturat" cand
`SIGN(CANTITATE) * (CANTITATE NVL(facturat,0)) > 0`.
Daca factura manuala a reprezentat liniile **altfel** decat comanda — tipic discounturi
comasate (ex. discounturile pe cote de TVA 11%/21% puse intr-o singura linie la 0% TVA) —
preturile nu se mai potrivesc, deci ROA arata comanda **tot nefacturata** desi headerul
e legat si dashboard-ul o vede facturata.
Scriptul **prezice** acest reziduu inainte de `--apply` (functia `order_line_residual`,
simuland factura ce urmeaza a fi legata) si il **re-verifica** dupa legare. Cand exista,
afiseaza `!! ATENTIE ...` cu liniile reziduale (ART / cantitate comanda / pret / facturat)
si un contor in rezumat. **Scriptul NU atinge `COMENZI_ELEMENTE`** — aceste cazuri se
corecteaza **manual in ROA** (aliniezi liniile comenzii la factura, ex. comasezi liniile
de discount ca in factura, pastrand valoarea totala).
## Utilizare
Ruleaza **pe serverul de productie VENDING** (are nevoie de Oracle prod +
`api/data/import.db` prod). Foloseste `app.config.settings` (deci `.env`-ul prod).
```bat
REM din C:\gomag-vending\scripts
relink_manual_invoices.bat REM dry-run, ultimele 3 zile (NU modifica)
relink_manual_invoices.bat --apply REM aplica, cu confirmare
relink_manual_invoices.bat --apply --yes REM aplica fara confirmare
relink_manual_invoices.bat --days 7 REM alta fereastra
```
Dublu-click pe `.bat` = dry-run. `.bat`-ul seteaza mediul Oracle thick-mode
(`TNS_ADMIN` + PATH instant client) ca `start.bat`.
Direct cu Python (echivalent):
```bat
set TNS_ADMIN=C:\roa\instantclient_11_2_0_2
set PATH=C:\app\Server\product\18.0.0\dbhomeXE\bin;%PATH%
C:\gomag-vending\venv\Scripts\python.exe scripts\relink_manual_invoices.py --apply
```
Din containerul de dev, peste SSH:
```bash
scp -P 22122 scripts/relink_manual_invoices.* gomag@79.119.86.134:C:/gomag-vending/scripts/
ssh -p 22122 gomag@79.119.86.134 'cmd /c "C:\gomag-vending\scripts\relink_manual_invoices.bat --days 3 < nul"'
```
**Workflow recomandat:** intai dry-run → verifica lista `LINK` si `SKIP_AMBIGUOUS`
→ apoi `--apply`. Cazurile `SKIP_AMBIGUOUS` se rezolva manual in ROA.
## Istoric
Codifica reconcilierea din **2026-06-26** (pana de curent la VENDING): pool cazut
~09:0710:25; 12 facturi manuale `TIP=1` (IDV 138191138203 → comenzi 54195430)
legate; 3 facturi de depozit corect excluse (CRISS VENDING, COFEE SEVEN TO GO,
PANDELE MIOARA); 2 parteneri duplicati semnalati (CERBU, MILITARU).
**Follow-up 2026-06-26 (reziduu de linie):** 2 din cele 12 comenzi (5419/web 492710430,
5423/web 492710513) au ramas nefacturate **in ROA** desi headerul era legat — factura
manuala comasase cele 2 linii de discount (ART 2077, split pe TVA 11%/21%) intr-una la
0% TVA, deci nu se potriveau pe `ID_ARTICOL+PRET`. Reparate manual prin alinierea
liniilor comenzii la factura (comasare in `COMENZI_ELEMENTE`, valoare discount pastrata).
De aici provine verificarea de reziduu de linie adaugata in script.
Vezi si: [oracle-schema-notes.md](oracle-schema-notes.md) (tabele `COMENZI`/`VANZARI`),
sectiunea „Facturi & Cache" din [README](../README.md).

102
scripts/db_maintenance.py Normal file
View File

@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""
One-shot SQLite + log maintenance, invoked by deploy.ps1 while the GoMagVending
service is stopped.
What it does:
1. Prune audit history older than --history-days (sync_runs, sync_run_orders,
orphaned sync_phase_failures). Business tables (orders, order_items) are
NEVER touched.
2. Enable PRAGMA auto_vacuum=INCREMENTAL and run a full VACUUM to reclaim disk.
3. Delete log files older than --log-days from logs/.
Plain sqlite3 only — no app imports, no Oracle, no event loop — so it runs even
if the app/Oracle env isn't set up.
Usage:
python scripts/db_maintenance.py # defaults: 7/7 days
python scripts/db_maintenance.py --history-days 7 --log-days 7
python scripts/db_maintenance.py --db C:\\path\\import.db
"""
import argparse
import os
import sqlite3
import sys
import time
from datetime import datetime, timedelta
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEFAULT_DB = os.path.join(REPO_ROOT, "api", "data", "import.db")
DEFAULT_LOGS = os.path.join(REPO_ROOT, "logs")
def prune_and_vacuum(db_path: str, history_days: int) -> None:
cutoff = (datetime.now() - timedelta(days=history_days)).strftime("%Y-%m-%d")
before = os.path.getsize(db_path) / 1048576.0
conn = sqlite3.connect(db_path, timeout=120)
try:
cur = conn.cursor()
cur.execute(
"DELETE FROM sync_run_orders WHERE sync_run_id IN "
"(SELECT run_id FROM sync_runs WHERE substr(started_at,1,10) < ?)",
(cutoff,))
junction = cur.rowcount
cur.execute(
"DELETE FROM sync_runs WHERE substr(started_at,1,10) < ?", (cutoff,))
runs = cur.rowcount
cur.execute(
"DELETE FROM sync_phase_failures "
"WHERE run_id NOT IN (SELECT run_id FROM sync_runs)")
conn.commit()
# auto_vacuum mode change only takes effect on the next VACUUM.
conn.isolation_level = None
conn.execute("PRAGMA auto_vacuum=INCREMENTAL")
t0 = time.time()
conn.execute("VACUUM")
vac = time.time() - t0
finally:
conn.close()
after = os.path.getsize(db_path) / 1048576.0
print(f"[db_maintenance] cutoff<{cutoff} runs_deleted={runs} "
f"junction_deleted={junction} size {before:.1f}MB -> {after:.1f}MB "
f"(VACUUM {vac:.1f}s)")
def cleanup_logs(log_dir: str, log_days: int) -> None:
if not os.path.isdir(log_dir):
print(f"[db_maintenance] logs dir not found: {log_dir}")
return
cutoff = time.time() - log_days * 86400
removed = 0
for name in os.listdir(log_dir):
if ".log" not in name:
continue
path = os.path.join(log_dir, name)
try:
if os.path.isfile(path) and os.path.getmtime(path) < cutoff:
os.remove(path)
removed += 1
except OSError as e:
print(f"[db_maintenance] could not remove {name}: {e}")
print(f"[db_maintenance] removed {removed} log file(s) older than {log_days}d")
def main() -> int:
ap = argparse.ArgumentParser(description="SQLite + log maintenance")
ap.add_argument("--db", default=DEFAULT_DB)
ap.add_argument("--logs-dir", default=DEFAULT_LOGS)
ap.add_argument("--history-days", type=int, default=7)
ap.add_argument("--log-days", type=int, default=7)
args = ap.parse_args()
if not os.path.exists(args.db):
# Non-fatal: a fresh install may not have a DB yet.
print(f"[db_maintenance] DB not found, skipping: {args.db}", file=sys.stderr)
else:
prune_and_vacuum(args.db, args.history_days)
cleanup_logs(args.logs_dir, args.log_days)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,38 @@
@echo off
REM ============================================================================
REM Reconciliere facturi manuale ROA <-> comenzi GoMag (relink VANZARI.ID_COMANDA)
REM Ruleaza pe serverul de productie VENDING. Seteaza mediul Oracle (thick mode)
REM exact ca start.bat, apoi apeleaza scriptul Python.
REM
REM Utilizare (dublu-click = dry-run, sau din cmd):
REM relink_manual_invoices.bat -> dry-run (ultimele 3 zile)
REM relink_manual_invoices.bat --apply -> aplica (cu confirmare)
REM relink_manual_invoices.bat --apply --yes -> aplica fara confirmare
REM relink_manual_invoices.bat --days 7 -> alta fereastra
REM relink_manual_invoices.bat --apply --days 7
REM ============================================================================
setlocal
REM --- Mediu Oracle (vezi start.bat) ---
set "TNS_ADMIN=C:\roa\instantclient_11_2_0_2"
set "PATH=C:\app\Server\product\18.0.0\dbhomeXE\bin;%PATH%"
set "PYTHONIOENCODING=utf-8"
REM --- Cai relative la acest .bat (scripts\) ---
set "PYEXE=%~dp0..\venv\Scripts\python.exe"
set "PYSCRIPT=%~dp0relink_manual_invoices.py"
if not exist "%PYEXE%" (
echo [EROARE] Nu gasesc venv-ul: "%PYEXE%"
echo Ruleaza din C:\gomag-vending\scripts pe serverul VENDING.
pause
exit /b 1
)
"%PYEXE%" "%PYSCRIPT%" %*
set "RC=%ERRORLEVEL%"
echo.
echo (cod iesire: %RC%)
pause
endlocal & exit /b %RC%

View File

@@ -0,0 +1,410 @@
#!/usr/bin/env python3
"""
Reconcile manual ROA invoices with GoMag orders left "nefacturate".
Context
-------
When the Oracle pool / sync is down (e.g. after a power loss) the warehouse
operator emits invoices MANUALLY in ROA. Those land in `VANZARI` with
`ID_COMANDA = NULL` (vs the app's normal flow which sets `ID_COMANDA` and links
to `COMENZI.COMANDA_EXTERNA` = GoMag order no). Once the app recovers it imports
the same web orders into `COMENZI`, but the manual invoice is never linked, so
the order stays "nefacturat" in ROA and in the dashboard.
This script finds those orphan invoices (`VANZARI.ID_COMANDA IS NULL`, `sters=0`)
and links each to its GoMag order, matching by **exact total + partner/name**,
then populates the SQLite invoice cache (`orders.factura_*`) exactly like the app.
IMPORTANT — warehouse / walk-in invoices
-----------------------------------------
The operator ALSO emits genuine manual invoices directly from the warehouse,
with NO online order behind them (~20+/day). Those have no matching uninvoiced
GoMag order, so they get classified SKIP_NOMATCH and are LEFT UNTOUCHED. The
matching is deliberately conservative: anything not an unambiguous 1:1 match is
reported for manual review, never auto-linked.
Run it ON the production server (it needs prod Oracle + prod import.db):
# dry-run (default) — shows the plan, changes nothing
C:\\gomag-vending\\venv\\Scripts\\python.exe scripts\\relink_manual_invoices.py
# apply, with confirmation
... scripts\\relink_manual_invoices.py --apply
# apply without confirmation (automation)
... scripts\\relink_manual_invoices.py --apply --yes
# widen / narrow the lookback window (default: last 3 days)
... scripts\\relink_manual_invoices.py --days 5
From the dev container you can drive it over SSH:
scp -P 22122 scripts/relink_manual_invoices.py gomag@79.119.86.134:C:/gomag-vending/scripts/
ssh -p 22122 gomag@79.119.86.134 "cd C:\\gomag-vending\\api; \
$env:TNS_ADMIN='C:\\roa\\instantclient_11_2_0_2'; \
C:\\gomag-vending\\venv\\Scripts\\python.exe ..\\scripts\\relink_manual_invoices.py"
"""
import argparse
import os
import re
import sqlite3
import sys
# Windows service console is cp1252; keep output robust regardless of code page.
try:
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
# Make the app package importable + load .env-backed settings (Oracle creds, SQLite path).
_API_ROOT = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "api")
sys.path.insert(0, _API_ROOT)
import oracledb # noqa: E402
from app.config import settings # noqa: E402
# Match tolerance for money comparison (lei). Totals are stored to 2 decimals.
MONEY_EPS = 0.01
# ─── Oracle ──────────────────────────────────────────────────────────────────
def oracle_connect():
if settings.TNS_ADMIN:
os.environ.setdefault("TNS_ADMIN", settings.TNS_ADMIN)
if settings.INSTANTCLIENTPATH:
try:
oracledb.init_oracle_client(lib_dir=settings.INSTANTCLIENTPATH)
except Exception:
pass # already initialized / thin mode
return oracledb.connect(
user=settings.ORACLE_USER,
password=settings.ORACLE_PASSWORD,
dsn=settings.ORACLE_DSN,
)
def fetch_orphan_invoices(cur, days):
"""Manual invoices with no order link, created in the lookback window."""
cur.execute(
"""
SELECT v.ID_VANZARE, v.ID_PART, v.SERIE_ACT, v.NUMAR_ACT,
v.TOTAL_FARA_TVA, v.TOTAL_TVA, v.TOTAL_CU_TVA,
TO_CHAR(v.DATA_ACT, 'YYYY-MM-DD') AS data_act,
TO_CHAR(v.DATAORA, 'YYYY-MM-DD HH24:MI') AS creat,
v.TIP, p.DENUMIRE
FROM VANZARI v
LEFT JOIN NOM_PARTENERI p ON p.ID_PART = v.ID_PART
WHERE v.STERS = 0
AND v.ID_COMANDA IS NULL
AND v.DATAORA >= TRUNC(SYSDATE) - :days
ORDER BY v.DATAORA
""",
days=days,
)
cols = [d[0].lower() for d in cur.description]
return [dict(zip(cols, r)) for r in cur.fetchall()]
def comanda_active(cur, id_comanda):
cur.execute("SELECT COUNT(*) FROM COMENZI WHERE ID_COMANDA = :1 AND STERS = 0", [id_comanda])
return cur.fetchone()[0] == 1
def comanda_already_invoiced(cur, id_comanda):
cur.execute(
"SELECT COUNT(*) FROM VANZARI WHERE ID_COMANDA = :1 AND STERS = 0", [id_comanda]
)
return cur.fetchone()[0] > 0
def order_line_residual(cur, id_comanda, extra_idv=None):
"""COMENZI_ELEMENTE lines NOT covered by the linked invoice(s), per ROA's own
line-level facturat test (`PACK_FACTURARE.cursor_comanda`): invoiced quantity is
matched to the order on **ID_ARTICOL + exact PRET**, and a line is "still to
invoice" when `SIGN(CANTITATE) * (CANTITATE - NVL(facturat, 0)) > 0`.
`extra_idv` simulates an invoice about to be linked, so the residual can be
PREDICTED before `--apply` (when VANZARI.ID_COMANDA is not set yet).
A non-empty result means linking the VANZARI header is NOT enough — ROA will
STILL show the order *nefacturat* (even though the app dashboard, which only
checks the header link, shows it facturat). Typical cause: the manual invoice
consolidated the order's discount lines (e.g. per-VAT-rate discounts merged into
one 0%-TVA line), so the prices no longer match the order's COMENZI_ELEMENTE.
Those need a manual line fix in ROA — the script never touches order lines.
"""
cur.execute(
"""
SELECT A.ID_COMANDA_ELEMENT, A.ID_ARTICOL, A.CANTITATE, A.PRET,
NVL(D.CANTITATE, 0) AS FACTURAT
FROM COMENZI_ELEMENTE A
LEFT JOIN (SELECT B1.ID_ARTICOL, B1.PRET, SUM(B1.CANTITATE) AS CANTITATE
FROM VANZARI A1
JOIN VANZARI_DETALII B1
ON A1.ID_VANZARE = B1.ID_VANZARE AND B1.STERS = 0
WHERE A1.STERS = 0
AND (A1.ID_COMANDA = :idc OR A1.ID_VANZARE = :idv)
GROUP BY B1.ID_ARTICOL, B1.PRET) D
ON A.ID_ARTICOL = D.ID_ARTICOL AND A.PRET = D.PRET
WHERE A.STERS = 0
AND A.ID_COMANDA = :idc
AND SIGN(A.CANTITATE) * (A.CANTITATE - NVL(D.CANTITATE, 0)) > 0
""",
idc=id_comanda, idv=extra_idv,
)
cols = [d[0].lower() for d in cur.description]
return [dict(zip(cols, r)) for r in cur.fetchall()]
# ─── SQLite ──────────────────────────────────────────────────────────────────
def fetch_uninvoiced_orders(db, days):
"""Imported GoMag orders that have an id_comanda but no cached invoice yet."""
cur = db.execute(
"""
SELECT order_number, id_comanda, id_partener, order_total,
customer_name, shipping_name, billing_name
FROM orders
WHERE status IN ('IMPORTED', 'ALREADY_IMPORTED')
AND id_comanda IS NOT NULL
AND (factura_numar IS NULL OR factura_numar = '')
AND order_date >= date('now', ?)
""",
(f'-{days} day',),
)
return [dict(r) for r in cur.fetchall()]
# ─── Name matching (handles duplicate partner records) ────────────────────────
_NAME_NOISE = re.compile(
r"\b(S\.?R\.?L\.?|S\.?C\.?|S\.?A\.?|P\.?F\.?A\.?|II|SRL|SC|SA)\b", re.IGNORECASE
)
def _tokens(name):
if not name:
return set()
name = _NAME_NOISE.sub(" ", name.upper())
name = re.sub(r"[^A-Z0-9 ]", " ", name)
return {t for t in name.split() if len(t) >= 3}
def name_match(a, b):
"""Conservative name overlap — tolerant of word order and SRL/SC noise."""
ta, tb = _tokens(a), _tokens(b)
if not ta or not tb:
return False
shared = ta & tb
return len(shared) >= 1 and len(shared) >= min(len(ta), len(tb)) * 0.5
def money_eq(a, b):
return a is not None and b is not None and abs(float(a) - float(b)) <= MONEY_EPS
# ─── Matching ─────────────────────────────────────────────────────────────────
def classify(inv, orders, cur):
"""Decide what to do with one orphan invoice.
Returns (action, order_or_None, note). action in:
LINK unambiguous match -> will link
SKIP_NOMATCH no uninvoiced GoMag order with this total -> warehouse/walk-in invoice
SKIP_AMBIGUOUS several plausible orders -> needs a human
SKIP_ALREADY matched comanda already has an invoice / is gone
"""
total = inv["total_cu_tva"]
cands = [o for o in orders if money_eq(o["order_total"], total)]
if not cands:
return ("SKIP_NOMATCH", None, "fara comanda online cu acest total (factura depozit)")
def pick(subset, why):
o = subset[0]
if not comanda_active(cur, o["id_comanda"]):
return ("SKIP_ALREADY", None, f"comanda {o['id_comanda']} nu mai e activa in ROA")
if comanda_already_invoiced(cur, o["id_comanda"]):
return ("SKIP_ALREADY", None, f"comanda {o['id_comanda']} are deja factura")
return ("LINK", o, why)
by_partner = [o for o in cands if o["id_partener"] == inv["id_part"]]
by_name = [
o for o in cands
if name_match(inv["denumire"], o["customer_name"])
or name_match(inv["denumire"], o["shipping_name"])
or name_match(inv["denumire"], o["billing_name"])
]
if len(by_partner) == 1:
return pick(by_partner, "potrivire partener+total")
if len(by_partner) > 1:
return ("SKIP_AMBIGUOUS", None,
f"{len(by_partner)} comenzi acelasi partener+total: "
+ ", ".join(o["order_number"] for o in by_partner))
if len(by_name) == 1:
return pick(by_name, "potrivire nume+total (partener dublat)")
if len(by_name) > 1:
return ("SKIP_AMBIGUOUS", None,
f"{len(by_name)} comenzi nume+total: "
+ ", ".join(o["order_number"] for o in by_name))
if len(cands) == 1:
return ("SKIP_AMBIGUOUS", None,
f"total se potriveste cu {cands[0]['order_number']} dar partenerul si numele difera")
return ("SKIP_AMBIGUOUS", None,
f"{len(cands)} comenzi cu acelasi total, niciun partener/nume sigur")
# ─── Apply ────────────────────────────────────────────────────────────────────
def apply_link(ora_cur, db, inv, order):
"""Link VANZARI -> COMANDA in Oracle and cache the invoice onto the SQLite order."""
ora_cur.execute(
"UPDATE VANZARI SET ID_COMANDA = :1 "
"WHERE ID_VANZARE = :2 AND ID_COMANDA IS NULL AND STERS = 0",
[order["id_comanda"], inv["id_vanzare"]],
)
linked = ora_cur.rowcount == 1
if linked:
numar = int(inv["numar_act"]) if inv["numar_act"] is not None else None
db.execute(
"""
UPDATE orders SET
factura_serie = ?, factura_numar = ?,
factura_total_fara_tva = ?, factura_total_tva = ?, factura_total_cu_tva = ?,
factura_data = ?, invoice_checked_at = datetime('now'),
updated_at = datetime('now')
WHERE order_number = ? AND (factura_numar IS NULL OR factura_numar = '')
""",
(inv["serie_act"], numar,
float(inv["total_fara_tva"] or 0), float(inv["total_tva"] or 0),
float(inv["total_cu_tva"] or 0), inv["data_act"], order["order_number"]),
)
return linked
# ─── Main ─────────────────────────────────────────────────────────────────────
def main():
ap = argparse.ArgumentParser(description="Relink manual ROA invoices to GoMag orders.")
ap.add_argument("--apply", action="store_true", help="apply changes (default: dry-run)")
ap.add_argument("--yes", action="store_true", help="skip confirmation prompt")
ap.add_argument("--days", type=int, default=3, help="lookback window in days (default 3)")
args = ap.parse_args()
conn = oracle_connect()
ora_cur = conn.cursor()
db = sqlite3.connect(settings.SQLITE_DB_PATH)
db.row_factory = sqlite3.Row
invoices = fetch_orphan_invoices(ora_cur, args.days)
orders = fetch_uninvoiced_orders(db, args.days)
print(f"Fereastra: ultimele {args.days} zile")
print(f"Facturi orfane (VANZARI ID_COMANDA NULL, sters=0): {len(invoices)}")
print(f"Comenzi GoMag nefacturate (cu id_comanda): {len(orders)}\n")
plans = []
for inv in invoices:
action, order, note = classify(inv, orders, ora_cur)
plans.append((inv, action, order, note))
# an order can only back one invoice
if action == "LINK" and order is not None:
orders = [o for o in orders if o["order_number"] != order["order_number"]]
# Predict ROA's line-level residual for each LINK. Linking the VANZARI header is
# not always enough: if the manual invoice represented the lines differently than
# the order (e.g. consolidated discounts), ROA still shows the order nefacturat.
residuals = {}
for inv, action, order, _note in plans:
if action == "LINK" and order is not None:
res = order_line_residual(ora_cur, order["id_comanda"], inv["id_vanzare"])
if res:
residuals[order["order_number"]] = res
def show(action, detailed=True):
rows = [(i, o, n) for (i, a, o, n) in plans if a == action]
if not rows:
return
print(f"-- {action} ({len(rows)}) --")
if not detailed:
print(" (facturi de depozit, fara comanda online — lasate neatinse)\n")
return
for inv, order, note in rows:
tag = f"-> {order['order_number']} (idcom {order['id_comanda']})" if order else ""
print(f" IDV={inv['id_vanzare']} {inv['serie_act']}{inv['numar_act']} "
f"tot={inv['total_cu_tva']} [{inv['denumire']}] {tag} {note}")
res = residuals.get(order["order_number"]) if order else None
if res:
print(f" !! ATENTIE: dupa legare ROA va arata comanda tot NEFACTURATA "
f"({len(res)} linii reziduale la nivel de element — factura nu le acopera "
f"pe ID_ARTICOL+PRET; probabil discount comasat/0% TVA). Necesita corectie "
f"manuala a liniilor in ROA:")
for r in res:
print(f" ART={r['id_articol']} CANT_COMANDA={r['cantitate']} "
f"PRET={r['pret']} facturat={r['facturat']}")
print()
for a in ("LINK", "SKIP_AMBIGUOUS", "SKIP_ALREADY"):
show(a)
show("SKIP_NOMATCH", detailed=False)
to_link = [(i, o) for (i, a, o, n) in plans if a == "LINK"]
ambiguous = sum(1 for (_, a, _, _) in plans if a == "SKIP_AMBIGUOUS")
with_residual = sum(1 for (_, o) in to_link if o and o["order_number"] in residuals)
print(f"De legat: {len(to_link)} | De verificat manual (AMBIGUOUS): {ambiguous} | "
f"Neatinse (depozit): {sum(1 for (_, a, _, _) in plans if a == 'SKIP_NOMATCH')}")
if with_residual:
print(f"!! Din care {with_residual} raman NEFACTURATE in ROA dupa legare "
f"(reziduu de linie — vezi ATENTIE mai sus; necesita corectie manuala a liniilor).")
if not args.apply:
print("\n[DRY-RUN] nimic modificat. Reruleaza cu --apply ca sa aplici.")
return
if not to_link:
print("\nNimic de legat.")
return
if not args.yes:
resp = input(f"\nAplici {len(to_link)} legaturi? [y/N] ").strip().lower()
if resp != "y":
print("Anulat.")
return
linked = 0
for inv, order in to_link:
if apply_link(ora_cur, db, inv, order):
linked += 1
print(f" OK IDV={inv['id_vanzare']} -> idcom {order['id_comanda']} "
f"({order['order_number']})")
else:
print(f" SKIP IDV={inv['id_vanzare']} — ID_COMANDA nu mai era NULL (concurenta)")
conn.commit()
db.commit()
print(f"\nAplicat: {linked} facturi legate + cache SQLite actualizat.")
# Verifica reziduul REAL dupa legare. Daca > 0, ROA arata comanda tot nefacturata
# desi headerul e legat (app dashboard o vede facturata). Liniile trebuie corectate
# manual in ROA — scriptul nu atinge niciodata COMENZI_ELEMENTE.
still = []
for inv, order in to_link:
res = order_line_residual(ora_cur, order["id_comanda"])
if res:
still.append((order, res))
if still:
print(f"\n!! ATENTIE — {len(still)} comenzi legate dar cu reziduu de linie in ROA "
f"(raman NEFACTURATE pana corectezi liniile manual in ROA):")
for order, res in still:
print(f" {order['order_number']} (idcom {order['id_comanda']}): {len(res)} linii — "
+ ", ".join(f"ART={r['id_articol']}@{r['pret']}" for r in res))
conn.close()
db.close()
if __name__ == "__main__":
main()