Compare commits

5 Commits

Author SHA1 Message Date
Claude Agent
371c73f5cf fix(deploy.ps1): strip non-ASCII so Windows PowerShell parses it
The file was UTF-8 without BOM; Windows PowerShell 5.1 reads it as CP1252,
where the box-drawing/em-dash/arrow bytes (0x94/0x92) decode to smart quotes
(U+201D/U+2019) that PS treats as string delimiters, breaking the parser
("string is missing the terminator"). Replaced -, =, em/en dash, arrows and
smart quotes with ASCII equivalents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:18:55 +00:00
Claude Agent
3b56029cf2 Merge feat/db-log-maintenance: guard DB + log growth (prune + Option B + rotation) 2026-06-26 10:05:18 +00:00
Claude Agent
dcc5042586 feat(maintenance): guard DB + log growth (Option B + daily prune + rotation)
Root cause of the 2GB prod import.db: the sync_run_orders audit junction
recorded every order on every run; under the 1-minute scheduler ~98% of
21.7M rows were no-op ALREADY_IMPORTED re-observations. NSSM stdout/stderr
also grew unbounded (rotation never applied to the live service).

Changes:
- sqlite_service: skip ALREADY_IMPORTED rows in sync_run_orders (write-side
  guard, _SKIP_JUNCTION_STATUSES); add prune_sync_history(retention_days)
  with incremental_vacuum.
- maintenance_service (new): cleanup_old_logs + run_daily_maintenance.
- scheduler_service: start_maintenance_job (daily CronTrigger).
- main.py: RotatingFileHandler (sync_comenzi_current.log, 10MB x5) instead
  of a new timestamped file per start; schedule daily maintenance + one-shot
  catch-up at startup.
- scripts/db_maintenance.py (new): one-shot prune + VACUUM + log cleanup,
  plain sqlite3, invoked by deploy.ps1 while the service is stopped.
- deploy.ps1: stop -> run db_maintenance.py -> (re)apply NSSM AppRotate*
  idempotently -> start, so rotation reaches pre-existing services.

Retention defaults: 7 days history, 7 days logs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:59:41 +00:00
Claude Agent
ccc6a933fa feat(scripts): line-level residual check in relink_manual_invoices
Linking the VANZARI header (ID_COMANDA) makes the app dashboard show an
order facturat, but ROA decides facturat at the LINE level
(PACK_FACTURARE.cursor_comanda matches invoiced qty on ID_ARTICOL + exact
PRET). When a manual invoice represented lines differently than the order
(e.g. per-VAT-rate discounts consolidated into one 0%-TVA line), the order
stays nefacturat in ROA despite the header link.

Add order_line_residual(): predicts the residual before --apply (via
extra_idv) and re-verifies after linking. Warns in the plan, the summary
counter, and post-apply when an order will still show nefacturat. The
script never touches COMENZI_ELEMENTE — those need a manual line fix.

Surfaced by orders 5419/5423 (web 492710430/492710513) on 2026-06-26.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 09:46:59 +00:00
Claude Agent
698d036de9 docs: ghid reconciliere facturi manuale ROA <-> comenzi GoMag
Documenteaza relink_manual_invoices.py/.bat: cand se foloseste, cauza tehnica
(VANZARI.ID_COMANDA NULL), logica de matching conservatoare, clasificarea
LINK/SKIP_*, exceptia facturilor de depozit, si workflow-ul de rulare pe prod.
Link in tabelul Documentatie Tehnica din README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:37:19 +00:00
9 changed files with 579 additions and 66 deletions

View File

@@ -517,6 +517,7 @@ ssh -i ~/.ssh/id_ed25519 -p 22122 gomag@79.119.86.134 \
| [docs/oracle-schema-notes.md](docs/oracle-schema-notes.md) | Schema Oracle: tabele comenzi, facturi, preturi, proceduri cheie |
| [docs/pack_facturare_analysis.md](docs/pack_facturare_analysis.md) | Analiza flow facturare: call chain, parametri, STOC lookup, FACT-008 |
| [docs/cautare-selectie-client-cod-fiscal-anaf.md](docs/cautare-selectie-client-cod-fiscal-anaf.md) | Cautare/selectie client dupa cod fiscal: normalizare CUI, verificare ANAF, gate, proceduri Oracle |
| [docs/relink-facturi-manuale.md](docs/relink-facturi-manuale.md) | Reconciliere facturi manuale ROA ↔ comenzi GoMag dupa downtime (`relink_manual_invoices.py/.bat`) |
| [scripts/HANDOFF_MAPPING.md](scripts/HANDOFF_MAPPING.md) | Matching GoMag SKU → ROA articole (strategie si rezultate) |
---

View File

@@ -1,9 +1,10 @@
import asyncio
from contextlib import asynccontextmanager
from datetime import datetime
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from pathlib import Path
import logging
import logging.handlers
import os
from .config import settings
@@ -19,8 +20,12 @@ _stream_handler.setFormatter(_formatter)
_log_dir = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'logs')
os.makedirs(_log_dir, exist_ok=True)
_log_filename = f"sync_comenzi_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log"
_file_handler = logging.FileHandler(os.path.join(_log_dir, _log_filename), encoding='utf-8')
# Rotating handler (10MB x 5 backups) instead of a new timestamped file per
# start — caps log growth and stops file proliferation across restarts. Fixed
# name still matches the QA glob `sync_comenzi_*.log`.
_file_handler = logging.handlers.RotatingFileHandler(
os.path.join(_log_dir, "sync_comenzi_current.log"),
maxBytes=10 * 1024 * 1024, backupCount=5, encoding='utf-8')
_file_handler.setFormatter(_formatter)
_root_logger = logging.getLogger()
@@ -54,6 +59,15 @@ async def lifespan(app: FastAPI):
except Exception:
pass
# Daily DB/log maintenance (prune audit history + cleanup old logs) + a
# one-shot catch-up so a long-down service reclaims immediately on start.
try:
from .services import maintenance_service
scheduler_service.start_maintenance_job()
asyncio.create_task(maintenance_service.run_daily_maintenance())
except Exception as e:
logger.warning(f"Maintenance scheduling failed: {e}")
logger.info("GoMag Import Manager started")
yield

View File

@@ -0,0 +1,75 @@
"""Periodic maintenance: prune audit history + clean up old log files.
Keeps the SQLite DB and the logs/ directory from growing unbounded. The audit
tables (sync_runs, sync_run_orders) were the only DB growth source under the
1-minute scheduler; business tables (orders, order_items) are never touched.
The one-shot heavy reclaim (full VACUUM, run while the service is stopped) lives
in scripts/db_maintenance.py and is invoked by deploy.ps1.
"""
import logging
import os
import time
logger = logging.getLogger(__name__)
DEFAULT_HISTORY_RETENTION_DAYS = 7
DEFAULT_LOG_RETENTION_DAYS = 7
def _logs_dir() -> str:
"""Absolute path to the repo-root logs/ directory (matches main.py)."""
here = os.path.dirname(os.path.abspath(__file__))
return os.path.join(os.path.abspath(os.path.join(here, "..", "..", "..")), "logs")
def cleanup_old_logs(retention_days: int = DEFAULT_LOG_RETENTION_DAYS,
log_dir: str | None = None) -> int:
"""Delete log files older than `retention_days`. Returns count removed.
Targets any file with `.log` in its name (covers `sync_comenzi_current.log`,
NSSM `service_stdout.log`, and rotated backups like `*.log.3`). The live
rotating files stay fresh (recent mtime) so they fall inside the window.
"""
log_dir = log_dir or _logs_dir()
if not os.path.isdir(log_dir):
return 0
cutoff = time.time() - retention_days * 86400
removed = 0
for name in os.listdir(log_dir):
if ".log" not in name:
continue
path = os.path.join(log_dir, name)
try:
if os.path.isfile(path) and os.path.getmtime(path) < cutoff:
os.remove(path)
removed += 1
except OSError as e:
logger.warning(f"cleanup_old_logs: could not remove {name}: {e}")
if removed:
logger.info(f"cleanup_old_logs: removed {removed} file(s) older than "
f"{retention_days}d from {log_dir}")
return removed
async def run_daily_maintenance(
history_days: int = DEFAULT_HISTORY_RETENTION_DAYS,
log_days: int = DEFAULT_LOG_RETENTION_DAYS) -> dict:
"""Daily job: prune audit history (+reclaim pages) and clean old log files.
Each step is isolated — a failure in one does not skip the other.
"""
from . import sqlite_service
result: dict = {}
try:
result["db"] = await sqlite_service.prune_sync_history(history_days)
except Exception as e:
logger.warning(f"run_daily_maintenance: prune_sync_history failed: {e}")
result["db_error"] = str(e)
try:
result["logs_removed"] = cleanup_old_logs(log_days)
except Exception as e:
logger.warning(f"run_daily_maintenance: cleanup_old_logs failed: {e}")
result["logs_error"] = str(e)
return result

View File

@@ -1,6 +1,7 @@
import logging
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.interval import IntervalTrigger
from apscheduler.triggers.cron import CronTrigger
logger = logging.getLogger(__name__)
@@ -42,6 +43,31 @@ def start_scheduler(interval_minutes: int = 10):
logger.info(f"Scheduler started with interval {interval_minutes}min")
def start_maintenance_job(hour: int = 3):
"""Schedule the daily DB/log maintenance job (prune history + cleanup logs).
Runs independently of the sync job — starts the scheduler if it isn't already
running so maintenance happens even when auto-sync is disabled.
"""
if _scheduler is None:
init_scheduler()
from . import maintenance_service
_scheduler.add_job(
maintenance_service.run_daily_maintenance,
trigger=CronTrigger(hour=hour, minute=0),
id="maintenance_job",
name="Daily DB/Log Maintenance",
replace_existing=True
)
if not _scheduler.running:
_scheduler.start()
logger.info(f"Maintenance job scheduled daily at {hour:02d}:00")
def stop_scheduler():
"""Stop the scheduler."""
global _is_running

View File

@@ -2,7 +2,7 @@ import json
import logging
import logging.handlers
import os
from datetime import datetime
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
from ..database import get_sqlite, get_sqlite_sync
from ..constants import OrderStatus
@@ -114,6 +114,45 @@ async def update_sync_run(run_id: str, status: str, total_orders: int = 0,
await db.close()
async def prune_sync_history(retention_days: int = 7) -> dict:
"""Delete sync_runs + sync_run_orders older than `retention_days`.
Audit-only tables — `orders`/`order_items` (business data) are never touched.
Frees pages via incremental_vacuum (prod DB is auto_vacuum=INCREMENTAL after
the initial reclaim). Returns counts for logging. See _SKIP_JUNCTION_STATUSES
for the complementary write-side guard.
"""
cutoff = (datetime.now(_tz_bucharest).replace(tzinfo=None)
- timedelta(days=retention_days)).strftime("%Y-%m-%d")
db = await get_sqlite()
try:
cur = await db.execute(
"DELETE FROM sync_run_orders WHERE sync_run_id IN "
"(SELECT run_id FROM sync_runs WHERE substr(started_at,1,10) < ?)",
(cutoff,))
junction_deleted = cur.rowcount
cur = await db.execute(
"DELETE FROM sync_runs WHERE substr(started_at,1,10) < ?", (cutoff,))
runs_deleted = cur.rowcount
# Drop phase-failure rows orphaned by the run deletion.
await db.execute(
"DELETE FROM sync_phase_failures "
"WHERE run_id NOT IN (SELECT run_id FROM sync_runs)")
await db.commit()
try:
await db.execute("PRAGMA incremental_vacuum")
await db.commit()
except Exception as e: # auto_vacuum may be OFF on a fresh dev DB
logger.debug(f"prune_sync_history: incremental_vacuum skipped: {e}")
logger.info(
f"prune_sync_history: cutoff<{cutoff} runs_deleted={runs_deleted} "
f"junction_deleted={junction_deleted}")
return {"cutoff": cutoff, "runs_deleted": runs_deleted,
"junction_deleted": junction_deleted}
finally:
await db.close()
async def upsert_order(sync_run_id: str, order_number: str, order_date: str,
customer_name: str, status: str, id_comanda: int = None,
id_partener: int = None, error_message: str = None,
@@ -171,8 +210,28 @@ async def upsert_order(sync_run_id: str, order_number: str, order_date: str,
await db.close()
# Audit junction policy (DB-size guard):
# The sync_run_orders junction recorded EVERY order seen on EVERY run. Under the
# 1-minute scheduler, ~98% of rows were no-op ALREADY_IMPORTED re-observations,
# which grew the table to 21M+ rows / 2GB. We no longer record those: the order's
# current state still lives in `orders`; the junction now only lists orders a run
# actually touched (new / changed / skipped / errored / cancelled). Run-detail
# views therefore show only meaningful orders per run.
_SKIP_JUNCTION_STATUSES = {OrderStatus.ALREADY_IMPORTED.value}
def _record_in_junction(status_at_run: str) -> bool:
"""Whether this per-run status is worth persisting in sync_run_orders."""
return status_at_run not in _SKIP_JUNCTION_STATUSES
async def add_sync_run_order(sync_run_id: str, order_number: str, status_at_run: str):
"""Record that this run processed this order (junction table)."""
"""Record that this run processed this order (junction table).
No-op ALREADY_IMPORTED observations are skipped — see _SKIP_JUNCTION_STATUSES.
"""
if not _record_in_junction(status_at_run):
return
db = await get_sqlite()
try:
await db.execute("""
@@ -258,10 +317,16 @@ async def _insert_orders_only(db, orders: list[dict]):
if not orders:
return
await db.executemany(_ORDERS_UPSERT_SQL, [_orders_row(d) for d in orders])
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
[(d["sync_run_id"], d["order_number"], d.get("status_at_run", d["status"])) for d in orders],
)
junction_rows = [
(d["sync_run_id"], d["order_number"], d.get("status_at_run", d["status"]))
for d in orders
if _record_in_junction(d.get("status_at_run", d["status"]))
]
if junction_rows:
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
junction_rows,
)
async def _insert_valid_batch(db, orders: list[dict]):
@@ -273,10 +338,16 @@ async def _insert_valid_batch(db, orders: list[dict]):
if not orders:
return
await db.executemany(_ORDERS_UPSERT_SQL, [_orders_row(d) for d in orders])
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
[(d["sync_run_id"], d["order_number"], d["status_at_run"]) for d in orders],
)
junction_rows = [
(d["sync_run_id"], d["order_number"], d["status_at_run"])
for d in orders
if _record_in_junction(d["status_at_run"])
]
if junction_rows:
await db.executemany(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
junction_rows,
)
all_items: list[tuple] = []
order_numbers_with_items: set = set()
@@ -314,10 +385,11 @@ async def _insert_single_order(db, d: dict):
Caller wraps in SAVEPOINT so a per-row failure doesn't poison the batch.
"""
await db.execute(_ORDERS_UPSERT_SQL, _orders_row(d))
await db.execute(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
(d["sync_run_id"], d["order_number"], d["status_at_run"]),
)
if _record_in_junction(d["status_at_run"]):
await db.execute(
"INSERT OR IGNORE INTO sync_run_orders (sync_run_id, order_number, status_at_run) VALUES (?, ?, ?)",
(d["sync_run_id"], d["order_number"], d["status_at_run"]),
)
raw_items = d.get("items", [])
if raw_items:
await db.execute("DELETE FROM order_items WHERE order_number = ?", (d["order_number"],))

View File

@@ -35,9 +35,9 @@ param(
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# Helpers
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
function Write-Step { param([string]$msg) Write-Host "`n==> $msg" -ForegroundColor Cyan }
function Write-OK { param([string]$msg) Write-Host " [OK] $msg" -ForegroundColor Green }
function Write-Warn { param([string]$msg) Write-Host " [WARN] $msg" -ForegroundColor Yellow }
@@ -46,9 +46,9 @@ function Write-Info { param([string]$msg) Write-Host " $msg" -Foregroun
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 1. Citire token Gitea
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Citire token Gitea"
$TokenFile = Join-Path $ScriptDir ".gittoken"
@@ -71,9 +71,9 @@ $RepoUrl = if ($GitToken) {
"https://gitea.romfast.ro/romfast/gomag-vending.git"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 2. Git clone / pull
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Git clone / pull"
# Verifica git instalat
@@ -106,9 +106,9 @@ if (Test-Path (Join-Path $RepoPath ".git")) {
Write-OK "git clone OK"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 3. Verificare Python
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Verificare Python"
$PythonCmd = $null
@@ -135,9 +135,9 @@ if (-not $PythonCmd) {
exit 1
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 4. Creare venv si instalare dependinte
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Virtual environment + dependinte"
$VenvDir = Join-Path $RepoPath "venv"
@@ -170,9 +170,9 @@ if ($needInstall) {
Write-OK "Dependinte deja up-to-date"
}
# ─────────────────────────────────────────────────────────────────────────────
# 5. Detectare Oracle Home sugestie INSTANTCLIENTPATH
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 5. Detectare Oracle Home -> sugestie INSTANTCLIENTPATH
# -----------------------------------------------------------------------------
Write-Step "Detectare Oracle"
$OracleHome = $env:ORACLE_HOME
@@ -207,9 +207,9 @@ if ($OracleHome -and (Test-Path $OracleHome)) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 6. Creare .env din template daca lipseste
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Fisier configurare api\.env"
$EnvFile = Join-Path $RepoPath "api\.env"
@@ -241,9 +241,9 @@ if (-not (Test-Path $EnvFile)) {
Write-OK "api\.env exista deja"
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 7. Creare directoare necesare
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Directoare date"
foreach ($dir in @("data", "output", "logs")) {
@@ -256,9 +256,9 @@ foreach ($dir in @("data", "output", "logs")) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 8. Generare start.bat
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Generare start.bat"
$StartBat = Join-Path $RepoPath "start.bat"
@@ -298,13 +298,13 @@ echo Starting GoMag Import Manager on http://0.0.0.0:$Port (prefix /gomag)
Set-Content -Path $StartBat -Value $StartBatContent -Encoding UTF8
Write-OK "start.bat generat: $StartBat"
# ─────────────────────────────────────────────────────────────────────────────
# 9. IIS Verificare ARR + URL Rewrite
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 9. IIS - Verificare ARR + URL Rewrite
# -----------------------------------------------------------------------------
Write-Step "Verificare module IIS"
if ($SkipIIS) {
Write-Warn "SkipIIS activ configurare IIS sarita"
Write-Warn "SkipIIS activ - configurare IIS sarita"
} else {
$ArrPath = "$env:SystemRoot\System32\inetsrv\arr.dll"
$UrlRewritePath = "$env:SystemRoot\System32\inetsrv\rewrite.dll"
@@ -328,9 +328,9 @@ if ($SkipIIS) {
Write-Info "Sau: winget install Microsoft.URLRewrite"
}
# ─────────────────────────────────────────────────────────────────────────
# 10. Configurare IIS copiere web.config
# ─────────────────────────────────────────────────────────────────────────
# -------------------------------------------------------------------------
# 10. Configurare IIS - copiere web.config
# -------------------------------------------------------------------------
if ($ArrOk -and $UrlRwOk) {
Write-Step "Configurare IIS reverse proxy"
@@ -354,7 +354,7 @@ if ($SkipIIS) {
}
} catch {
Write-Warn "Nu am putut activa ARR proxy automat: $($_.Exception.Message)"
Write-Info "Activeaza manual din IIS Manager server root Application Request Routing Cache Enable Proxy"
Write-Info "Activeaza manual din IIS Manager -> server root -> Application Request Routing Cache -> Enable Proxy"
}
# Determina wwwroot site-ului IIS
@@ -406,13 +406,13 @@ if ($SkipIIS) {
}
}
} else {
Write-Warn "IIS nu e configurat complet instaleaza ARR si URL Rewrite, apoi ruleaza deploy.ps1 din nou"
Write-Warn "IIS nu e configurat complet - instaleaza ARR si URL Rewrite, apoi ruleaza deploy.ps1 din nou"
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# 11. Serviciu Windows (NSSM sau Task Scheduler)
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Step "Serviciu Windows"
$ServiceName = "GoMagVending"
@@ -431,26 +431,39 @@ if ($NssmExe) {
$existingService = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
if ($existingService) {
Write-Info "Serviciu existent, restarteaza..."
& $NssmExe restart $ServiceName
Write-OK "Serviciu $ServiceName restartat"
} else {
if (-not $existingService) {
Write-Info "Instalez serviciu $ServiceName cu NSSM..."
& $NssmExe install $ServiceName (Join-Path $RepoPath "start.bat")
& $NssmExe set $ServiceName AppDirectory $RepoPath
& $NssmExe set $ServiceName DisplayName "GoMag Vending Import Manager"
& $NssmExe set $ServiceName Description "Import comenzi web GoMag -> ROA Oracle"
& $NssmExe set $ServiceName Start SERVICE_AUTO_START
& $NssmExe set $ServiceName AppStdout (Join-Path $RepoPath "logs\service_stdout.log")
& $NssmExe set $ServiceName AppStderr (Join-Path $RepoPath "logs\service_stderr.log")
& $NssmExe set $ServiceName AppRotateFiles 1
& $NssmExe set $ServiceName AppRotateOnline 1
& $NssmExe set $ServiceName AppRotateBytes 10485760
& $NssmExe start $ServiceName
Write-OK "Serviciu $ServiceName instalat si pornit"
} else {
Write-Info "Serviciu existent, il opresc pentru mentenanta..."
& $NssmExe stop $ServiceName 2>$null
Start-Sleep -Seconds 3
}
# Mentenanta DB + log-uri cu serviciul oprit: prune istoric (7z) + VACUUM
# (reclaim disc) + cleanup log-uri (7z). Non-fatal daca esueaza.
Write-Info "Mentenanta DB/log-uri (prune + VACUUM + cleanup)..."
try {
& $VenvPy (Join-Path $RepoPath "scripts\db_maintenance.py") --history-days 7 --log-days 7
} catch {
Write-Warn "Mentenanta DB a esuat (continui): $_"
}
# (Re)aplica config log/rotatie de fiecare data - idempotent, astfel rotatia
# ajunge si pe serviciile instalate inainte ca aceste setari sa existe.
& $NssmExe set $ServiceName AppStdout (Join-Path $RepoPath "logs\service_stdout.log")
& $NssmExe set $ServiceName AppStderr (Join-Path $RepoPath "logs\service_stderr.log")
& $NssmExe set $ServiceName AppRotateFiles 1
& $NssmExe set $ServiceName AppRotateOnline 1
& $NssmExe set $ServiceName AppRotateBytes 10485760
& $NssmExe start $ServiceName
Write-OK "Serviciu $ServiceName pornit (mentenanta + rotatie aplicate)"
} else {
# Fallback: Task Scheduler
Write-Warn "NSSM nu este instalat"
@@ -498,13 +511,13 @@ if ($NssmExe) {
}
}
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
# Sumar final
# ─────────────────────────────────────────────────────────────────────────────
# -----------------------------------------------------------------------------
Write-Host ""
Write-Host "══════════════════════════════════════════════════════" -ForegroundColor Cyan
Write-Host " GoMag Vending Deploy Sumar" -ForegroundColor Cyan
Write-Host "══════════════════════════════════════════════════════" -ForegroundColor Cyan
Write-Host "======================================================" -ForegroundColor Cyan
Write-Host " GoMag Vending Deploy - Sumar" -ForegroundColor Cyan
Write-Host "======================================================" -ForegroundColor Cyan
Write-Host ""
Write-Host " Repo: $RepoPath" -ForegroundColor White
Write-Host " FastAPI: http://localhost:$Port/gomag" -ForegroundColor White
@@ -512,13 +525,13 @@ Write-Host " start.bat generat" -ForegroundColor White
Write-Host ""
if (-not (Test-Path $EnvFile)) {
Write-Host " [!] api\.env lipseste configureaza inainte de start!" -ForegroundColor Red
Write-Host " [!] api\.env lipseste - configureaza inainte de start!" -ForegroundColor Red
} else {
Write-Host " api\.env: OK" -ForegroundColor Green
# Verifica daca mai are valori placeholder
$envContent = Get-Content $EnvFile -Raw
if ($envContent -match "your_api_key_here|USER_ORACLE|parola_oracle|TNS_ALIAS") {
Write-Host " [!] api\.env contine valori placeholder editeaza!" -ForegroundColor Yellow
Write-Host " [!] api\.env contine valori placeholder - editeaza!" -ForegroundColor Yellow
}
}

View File

@@ -0,0 +1,133 @@
# Reconciliere facturi manuale ROA ↔ comenzi GoMag
**Script:** [`scripts/relink_manual_invoices.py`](../scripts/relink_manual_invoices.py)
**Launcher Windows:** [`scripts/relink_manual_invoices.bat`](../scripts/relink_manual_invoices.bat)
## Cand se foloseste
Cand Oracle pool / sync-ul **a fost cazut** (ex. dupa o pana de curent la VENDING)
si operatorul a emis **facturi manual direct in ROA** in acel interval. Dupa ce
aplicatia revine, importeaza aceleasi comenzi web in `COMENZI`, **dar** factura
manuala nu se leaga de comanda → comanda ramane **nefacturata** in ROA si in
dashboard, desi factura exista.
Semnal: in dashboard apar comenzi „nefacturate" pentru clienti care au fost de
fapt facturati, iar in `VANZARI` exista facturi cu `ID_COMANDA IS NULL`.
## Cauza tehnica
| Flux | `VANZARI.ID_COMANDA` | `COMENZI.COMANDA_EXTERNA` |
|------|----------------------|----------------------------|
| Normal (app) | setat (leaga factura de comanda) | nr comanda GoMag |
| Manual (operator, in downtime) | **NULL** | — |
Dashboard-ul / cache-ul SQLite marcheaza o comanda „Facturat" doar daca exista
`VANZARI ... WHERE id_comanda = <comanda> AND sters = 0`
(vezi `invoice_service.check_invoices_for_orders`). Factura manuala cu
`ID_COMANDA NULL` nu e niciodata gasita → comanda apare nefacturata.
`ID_FACT` (documentul fiscal) si `COMENZI.COMANDA_EXTERNA` sunt deja completate;
**singura piesa lipsa e legatura `VANZARI → COMANDA`.**
## ⚠️ Facturi de depozit (walk-in) — NU se ating
Operatorul emite zilnic si **facturi manuale legitime din depozit** (~20+/zi),
fara nicio comanda online in spate. Acestea au tot `ID_COMANDA NULL`, dar **nu**
trebuie legate de nimic. De aceea matching-ul e **conservator**: orice nu e o
potrivire 1:1 sigura e raportat, niciodata legat automat.
## Cum potriveste
Pentru fiecare factura orfana (`VANZARI.ID_COMANDA NULL`, `sters=0`, in fereastra
`--days`) cauta o comanda GoMag nefacturata (in SQLite, cu `id_comanda` setat) cu:
1. **Total identic** (`TOTAL_CU_TVA``order_total`, toleranta 0.01 lei), apoi
2. **acelasi partener** (`ID_PART` = `id_partener`) **SAU**
3. **nume potrivit** (token-overlap, tolereaza SRL/SC si ordinea cuvintelor) —
acopera cazul in care operatorul a creat un **partener duplicat** la facturare.
Verifica si ca acea comanda e activa in ROA (`sters=0`) si **nu** are deja factura
(anti-dubla-facturare).
### Clasificare (in raport)
| Eticheta | Ce inseamna | Actiune |
|----------|-------------|---------|
| `LINK` | potrivire 1:1 sigura | leaga (la `--apply`) |
| `SKIP_NOMATCH` | nicio comanda online cu acel total | **factura de depozit — lasata neatinsa** |
| `SKIP_AMBIGUOUS` | mai multe comenzi plauzibile, sau total potrivit dar partener+nume diferit | raportat pentru verificare manuala |
| `SKIP_ALREADY` | comanda nu mai e activa / are deja factura | sarit |
La aplicare: `UPDATE VANZARI SET ID_COMANDA = <comanda>` + populeaza
`orders.factura_*` in SQLite, exact ca aplicatia (`update_order_invoice`).
### ⚠ Verificare reziduu de linie (legatura header nu e mereu suficienta)
Aplicatia / dashboard-ul marcheaza o comanda „Facturat" doar dupa **legatura header**
(`VANZARI.ID_COMANDA`). **ROA insa verifica la nivel de linie**: in
`PACK_FACTURARE.cursor_comanda`, cantitatea facturata se potriveste cu comanda pe
**`ID_ARTICOL` + `PRET` exact**, iar o linie e „de facturat" cand
`SIGN(CANTITATE) * (CANTITATE NVL(facturat,0)) > 0`.
Daca factura manuala a reprezentat liniile **altfel** decat comanda — tipic discounturi
comasate (ex. discounturile pe cote de TVA 11%/21% puse intr-o singura linie la 0% TVA) —
preturile nu se mai potrivesc, deci ROA arata comanda **tot nefacturata** desi headerul
e legat si dashboard-ul o vede facturata.
Scriptul **prezice** acest reziduu inainte de `--apply` (functia `order_line_residual`,
simuland factura ce urmeaza a fi legata) si il **re-verifica** dupa legare. Cand exista,
afiseaza `!! ATENTIE ...` cu liniile reziduale (ART / cantitate comanda / pret / facturat)
si un contor in rezumat. **Scriptul NU atinge `COMENZI_ELEMENTE`** — aceste cazuri se
corecteaza **manual in ROA** (aliniezi liniile comenzii la factura, ex. comasezi liniile
de discount ca in factura, pastrand valoarea totala).
## Utilizare
Ruleaza **pe serverul de productie VENDING** (are nevoie de Oracle prod +
`api/data/import.db` prod). Foloseste `app.config.settings` (deci `.env`-ul prod).
```bat
REM din C:\gomag-vending\scripts
relink_manual_invoices.bat REM dry-run, ultimele 3 zile (NU modifica)
relink_manual_invoices.bat --apply REM aplica, cu confirmare
relink_manual_invoices.bat --apply --yes REM aplica fara confirmare
relink_manual_invoices.bat --days 7 REM alta fereastra
```
Dublu-click pe `.bat` = dry-run. `.bat`-ul seteaza mediul Oracle thick-mode
(`TNS_ADMIN` + PATH instant client) ca `start.bat`.
Direct cu Python (echivalent):
```bat
set TNS_ADMIN=C:\roa\instantclient_11_2_0_2
set PATH=C:\app\Server\product\18.0.0\dbhomeXE\bin;%PATH%
C:\gomag-vending\venv\Scripts\python.exe scripts\relink_manual_invoices.py --apply
```
Din containerul de dev, peste SSH:
```bash
scp -P 22122 scripts/relink_manual_invoices.* gomag@79.119.86.134:C:/gomag-vending/scripts/
ssh -p 22122 gomag@79.119.86.134 'cmd /c "C:\gomag-vending\scripts\relink_manual_invoices.bat --days 3 < nul"'
```
**Workflow recomandat:** intai dry-run → verifica lista `LINK` si `SKIP_AMBIGUOUS`
→ apoi `--apply`. Cazurile `SKIP_AMBIGUOUS` se rezolva manual in ROA.
## Istoric
Codifica reconcilierea din **2026-06-26** (pana de curent la VENDING): pool cazut
~09:0710:25; 12 facturi manuale `TIP=1` (IDV 138191138203 → comenzi 54195430)
legate; 3 facturi de depozit corect excluse (CRISS VENDING, COFEE SEVEN TO GO,
PANDELE MIOARA); 2 parteneri duplicati semnalati (CERBU, MILITARU).
**Follow-up 2026-06-26 (reziduu de linie):** 2 din cele 12 comenzi (5419/web 492710430,
5423/web 492710513) au ramas nefacturate **in ROA** desi headerul era legat — factura
manuala comasase cele 2 linii de discount (ART 2077, split pe TVA 11%/21%) intr-una la
0% TVA, deci nu se potriveau pe `ID_ARTICOL+PRET`. Reparate manual prin alinierea
liniilor comenzii la factura (comasare in `COMENZI_ELEMENTE`, valoare discount pastrata).
De aici provine verificarea de reziduu de linie adaugata in script.
Vezi si: [oracle-schema-notes.md](oracle-schema-notes.md) (tabele `COMENZI`/`VANZARI`),
sectiunea „Facturi & Cache" din [README](../README.md).

102
scripts/db_maintenance.py Normal file
View File

@@ -0,0 +1,102 @@
#!/usr/bin/env python3
"""
One-shot SQLite + log maintenance, invoked by deploy.ps1 while the GoMagVending
service is stopped.
What it does:
1. Prune audit history older than --history-days (sync_runs, sync_run_orders,
orphaned sync_phase_failures). Business tables (orders, order_items) are
NEVER touched.
2. Enable PRAGMA auto_vacuum=INCREMENTAL and run a full VACUUM to reclaim disk.
3. Delete log files older than --log-days from logs/.
Plain sqlite3 only — no app imports, no Oracle, no event loop — so it runs even
if the app/Oracle env isn't set up.
Usage:
python scripts/db_maintenance.py # defaults: 7/7 days
python scripts/db_maintenance.py --history-days 7 --log-days 7
python scripts/db_maintenance.py --db C:\\path\\import.db
"""
import argparse
import os
import sqlite3
import sys
import time
from datetime import datetime, timedelta
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEFAULT_DB = os.path.join(REPO_ROOT, "api", "data", "import.db")
DEFAULT_LOGS = os.path.join(REPO_ROOT, "logs")
def prune_and_vacuum(db_path: str, history_days: int) -> None:
cutoff = (datetime.now() - timedelta(days=history_days)).strftime("%Y-%m-%d")
before = os.path.getsize(db_path) / 1048576.0
conn = sqlite3.connect(db_path, timeout=120)
try:
cur = conn.cursor()
cur.execute(
"DELETE FROM sync_run_orders WHERE sync_run_id IN "
"(SELECT run_id FROM sync_runs WHERE substr(started_at,1,10) < ?)",
(cutoff,))
junction = cur.rowcount
cur.execute(
"DELETE FROM sync_runs WHERE substr(started_at,1,10) < ?", (cutoff,))
runs = cur.rowcount
cur.execute(
"DELETE FROM sync_phase_failures "
"WHERE run_id NOT IN (SELECT run_id FROM sync_runs)")
conn.commit()
# auto_vacuum mode change only takes effect on the next VACUUM.
conn.isolation_level = None
conn.execute("PRAGMA auto_vacuum=INCREMENTAL")
t0 = time.time()
conn.execute("VACUUM")
vac = time.time() - t0
finally:
conn.close()
after = os.path.getsize(db_path) / 1048576.0
print(f"[db_maintenance] cutoff<{cutoff} runs_deleted={runs} "
f"junction_deleted={junction} size {before:.1f}MB -> {after:.1f}MB "
f"(VACUUM {vac:.1f}s)")
def cleanup_logs(log_dir: str, log_days: int) -> None:
if not os.path.isdir(log_dir):
print(f"[db_maintenance] logs dir not found: {log_dir}")
return
cutoff = time.time() - log_days * 86400
removed = 0
for name in os.listdir(log_dir):
if ".log" not in name:
continue
path = os.path.join(log_dir, name)
try:
if os.path.isfile(path) and os.path.getmtime(path) < cutoff:
os.remove(path)
removed += 1
except OSError as e:
print(f"[db_maintenance] could not remove {name}: {e}")
print(f"[db_maintenance] removed {removed} log file(s) older than {log_days}d")
def main() -> int:
ap = argparse.ArgumentParser(description="SQLite + log maintenance")
ap.add_argument("--db", default=DEFAULT_DB)
ap.add_argument("--logs-dir", default=DEFAULT_LOGS)
ap.add_argument("--history-days", type=int, default=7)
ap.add_argument("--log-days", type=int, default=7)
args = ap.parse_args()
if not os.path.exists(args.db):
# Non-fatal: a fresh install may not have a DB yet.
print(f"[db_maintenance] DB not found, skipping: {args.db}", file=sys.stderr)
else:
prune_and_vacuum(args.db, args.history_days)
cleanup_logs(args.logs_dir, args.log_days)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -119,6 +119,45 @@ def comanda_already_invoiced(cur, id_comanda):
return cur.fetchone()[0] > 0
def order_line_residual(cur, id_comanda, extra_idv=None):
"""COMENZI_ELEMENTE lines NOT covered by the linked invoice(s), per ROA's own
line-level facturat test (`PACK_FACTURARE.cursor_comanda`): invoiced quantity is
matched to the order on **ID_ARTICOL + exact PRET**, and a line is "still to
invoice" when `SIGN(CANTITATE) * (CANTITATE - NVL(facturat, 0)) > 0`.
`extra_idv` simulates an invoice about to be linked, so the residual can be
PREDICTED before `--apply` (when VANZARI.ID_COMANDA is not set yet).
A non-empty result means linking the VANZARI header is NOT enough — ROA will
STILL show the order *nefacturat* (even though the app dashboard, which only
checks the header link, shows it facturat). Typical cause: the manual invoice
consolidated the order's discount lines (e.g. per-VAT-rate discounts merged into
one 0%-TVA line), so the prices no longer match the order's COMENZI_ELEMENTE.
Those need a manual line fix in ROA — the script never touches order lines.
"""
cur.execute(
"""
SELECT A.ID_COMANDA_ELEMENT, A.ID_ARTICOL, A.CANTITATE, A.PRET,
NVL(D.CANTITATE, 0) AS FACTURAT
FROM COMENZI_ELEMENTE A
LEFT JOIN (SELECT B1.ID_ARTICOL, B1.PRET, SUM(B1.CANTITATE) AS CANTITATE
FROM VANZARI A1
JOIN VANZARI_DETALII B1
ON A1.ID_VANZARE = B1.ID_VANZARE AND B1.STERS = 0
WHERE A1.STERS = 0
AND (A1.ID_COMANDA = :idc OR A1.ID_VANZARE = :idv)
GROUP BY B1.ID_ARTICOL, B1.PRET) D
ON A.ID_ARTICOL = D.ID_ARTICOL AND A.PRET = D.PRET
WHERE A.STERS = 0
AND A.ID_COMANDA = :idc
AND SIGN(A.CANTITATE) * (A.CANTITATE - NVL(D.CANTITATE, 0)) > 0
""",
idc=id_comanda, idv=extra_idv,
)
cols = [d[0].lower() for d in cur.description]
return [dict(zip(cols, r)) for r in cur.fetchall()]
# ─── SQLite ──────────────────────────────────────────────────────────────────
def fetch_uninvoiced_orders(db, days):
@@ -275,6 +314,16 @@ def main():
if action == "LINK" and order is not None:
orders = [o for o in orders if o["order_number"] != order["order_number"]]
# Predict ROA's line-level residual for each LINK. Linking the VANZARI header is
# not always enough: if the manual invoice represented the lines differently than
# the order (e.g. consolidated discounts), ROA still shows the order nefacturat.
residuals = {}
for inv, action, order, _note in plans:
if action == "LINK" and order is not None:
res = order_line_residual(ora_cur, order["id_comanda"], inv["id_vanzare"])
if res:
residuals[order["order_number"]] = res
def show(action, detailed=True):
rows = [(i, o, n) for (i, a, o, n) in plans if a == action]
if not rows:
@@ -287,6 +336,15 @@ def main():
tag = f"-> {order['order_number']} (idcom {order['id_comanda']})" if order else ""
print(f" IDV={inv['id_vanzare']} {inv['serie_act']}{inv['numar_act']} "
f"tot={inv['total_cu_tva']} [{inv['denumire']}] {tag} {note}")
res = residuals.get(order["order_number"]) if order else None
if res:
print(f" !! ATENTIE: dupa legare ROA va arata comanda tot NEFACTURATA "
f"({len(res)} linii reziduale la nivel de element — factura nu le acopera "
f"pe ID_ARTICOL+PRET; probabil discount comasat/0% TVA). Necesita corectie "
f"manuala a liniilor in ROA:")
for r in res:
print(f" ART={r['id_articol']} CANT_COMANDA={r['cantitate']} "
f"PRET={r['pret']} facturat={r['facturat']}")
print()
for a in ("LINK", "SKIP_AMBIGUOUS", "SKIP_ALREADY"):
@@ -295,8 +353,12 @@ def main():
to_link = [(i, o) for (i, a, o, n) in plans if a == "LINK"]
ambiguous = sum(1 for (_, a, _, _) in plans if a == "SKIP_AMBIGUOUS")
with_residual = sum(1 for (_, o) in to_link if o and o["order_number"] in residuals)
print(f"De legat: {len(to_link)} | De verificat manual (AMBIGUOUS): {ambiguous} | "
f"Neatinse (depozit): {sum(1 for (_, a, _, _) in plans if a == 'SKIP_NOMATCH')}")
if with_residual:
print(f"!! Din care {with_residual} raman NEFACTURATE in ROA dupa legare "
f"(reziduu de linie — vezi ATENTIE mai sus; necesita corectie manuala a liniilor).")
if not args.apply:
print("\n[DRY-RUN] nimic modificat. Reruleaza cu --apply ca sa aplici.")
@@ -325,6 +387,21 @@ def main():
db.commit()
print(f"\nAplicat: {linked} facturi legate + cache SQLite actualizat.")
# Verifica reziduul REAL dupa legare. Daca > 0, ROA arata comanda tot nefacturata
# desi headerul e legat (app dashboard o vede facturata). Liniile trebuie corectate
# manual in ROA — scriptul nu atinge niciodata COMENZI_ELEMENTE.
still = []
for inv, order in to_link:
res = order_line_residual(ora_cur, order["id_comanda"])
if res:
still.append((order, res))
if still:
print(f"\n!! ATENTIE — {len(still)} comenzi legate dar cu reziduu de linie in ROA "
f"(raman NEFACTURATE pana corectezi liniile manual in ROA):")
for order, res in still:
print(f" {order['order_number']} (idcom {order['id_comanda']}): {len(res)} linii — "
+ ", ".join(f"ART={r['id_articol']}@{r['pret']}" for r in res))
conn.close()
db.close()