Compare commits

...

2 Commits

Author SHA1 Message Date
Claude Agent
b930b2bc85 documentatie 2026-03-16 18:18:45 +00:00
Claude Agent
5dfd795908 fix(sync): detect deleted orders and invoices in ROA
Previously, orders deleted from Oracle (sters=1) remained as IMPORTED
in SQLite, and deleted invoices kept stale cache data. Now the refresh
button and sync cycle re-verify all imported orders against Oracle:
- Deleted orders → marked DELETED_IN_ROA with cleared id_comanda
- Deleted invoices → invoice cache fields cleared
- New status badge for DELETED_IN_ROA in dashboard and logs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 18:18:36 +00:00
8 changed files with 409 additions and 39 deletions

View File

@@ -97,6 +97,11 @@ screenshots/
[GoMag API] → [Python Sync Service] → [Oracle PL/SQL] → [FastAPI Admin]
↓ ↓ ↑ ↑
JSON Orders Download/Parse/Import Store/Update Dashboard + Config
[SQLite — tracking DB]
orders, sync_runs, missing_skus,
order_items, web_products,
invoice cache, app_settings
```
### FastAPI App Structure
@@ -106,6 +111,65 @@ screenshots/
- **Static:** CSS (`style.css`), JS (`shared.js`, `dashboard.js`, `logs.js`, `mappings.js`)
- **Databases:** Oracle (ERP data) + SQLite (order tracking, sync runs)
## API Endpoints — Sync & Comenzi
### Sync
| Method | Path | Descriere |
|--------|------|-----------|
| POST | `/api/sync/start` | Porneste sync in background |
| POST | `/api/sync/stop` | Trimite semnal de stop |
| GET | `/api/sync/status` | Status curent + last_run |
| GET | `/api/sync/history` | Istoric run-uri (paginat) |
| GET | `/api/sync/run/{run_id}` | Detalii run |
| GET | `/api/sync/run/{run_id}/log` | Log per comanda (JSON) |
| GET | `/api/sync/run/{run_id}/text-log` | Log text (live sau din SQLite) |
| GET | `/api/sync/run/{run_id}/orders` | Comenzi run filtrate/paginate |
| GET | `/api/sync/order/{order_number}` | Detaliu comanda + items + factura |
### Dashboard Comenzi
| Method | Path | Descriere |
|--------|------|-----------|
| GET | `/api/dashboard/orders` | Comenzi cu date factura (cache SQLite → Oracle fallback) |
| POST | `/api/dashboard/refresh-invoices` | Force-refresh stare facturi + comenzi sterse din ROA |
**Parametri `/api/dashboard/orders`:**
- `period_days`: 3/7/30/90 sau 0 (all / custom range)
- `period_start`, `period_end`: interval custom (cand `period_days=0`)
- `status`: `all` / `IMPORTED` / `SKIPPED` / `ERROR` / `UNINVOICED` / `INVOICED`
- `search`, `sort_by`, `sort_dir`, `page`, `per_page`
**`POST /api/dashboard/refresh-invoices` face:**
1. Necacturate → verifica Oracle daca au primit factura
2. Cacturate → verifica Oracle daca factura a fost stearsa
3. Toate importate → verifica Oracle daca comanda a fost stearsa (→ `DELETED_IN_ROA`)
### Scheduler
| Method | Path | Descriere |
|--------|------|-----------|
| PUT | `/api/sync/schedule` | Configureaza scheduler (enabled, interval_minutes) |
| GET | `/api/sync/schedule` | Status curent scheduler |
### Settings
| Method | Path | Descriere |
|--------|------|-----------|
| GET | `/api/settings` | Citeste setari aplicatie |
| PUT | `/api/settings` | Salveaza setari |
| GET | `/api/settings/sectii` | Lista sectii Oracle (dropdown) |
| GET | `/api/settings/politici` | Lista politici preturi Oracle (dropdown) |
## Invoice Cache (SQLite)
Facturile sunt cacate in coloana `factura_*` pe tabelul `orders`:
- `factura_serie`, `factura_numar`, `factura_data`
- `factura_total_fara_tva`, `factura_total_tva`, `factura_total_cu_tva`
**Sursa Oracle:** `SELECT ... FROM vanzari WHERE id_comanda IN (...) AND sters=0`
**Populare cache:**
- La fiecare cerere `/api/dashboard/orders` — comenzile fara cache sunt verificate live si cacate
- La deschidere detaliu comanda `/api/sync/order/{order_number}` — verifica live daca nu e caat
- La `POST /api/dashboard/refresh-invoices` — refresh complet pentru toate comenzile
## Business Rules
### Partners
@@ -124,6 +188,8 @@ screenshots/
- Default: ID_GESTIUNE=1, ID_SECTIE=1, ID_POL=0
- Delivery date = order date + 1 day
- All orders: INTERNA=0 (external)
- **Statuses:** `IMPORTED` / `ALREADY_IMPORTED` / `SKIPPED` / `ERROR` / `DELETED_IN_ROA`
- Upsert rule: daca status=`IMPORTED` exista, nu se suprascrie cu `ALREADY_IMPORTED`
## Configuration

137
README.md
View File

@@ -173,6 +173,18 @@ gomag-vending/
6. Rezultate salvate in SQLite (orders, sync_run_orders, order_items)
```
### Statuses Comenzi
| Status | Descriere |
|--------|-----------|
| `IMPORTED` | Importata nou in ROA in acest run |
| `ALREADY_IMPORTED` | Existenta deja in Oracle, contorizata |
| `SKIPPED` | SKU-uri lipsa → neimportata |
| `ERROR` | Eroare la import |
| `DELETED_IN_ROA` | A fost importata dar comanda a fost stearsa din ROA |
**Regula upsert:** daca statusul existent este `IMPORTED`, nu se suprascrie cu `ALREADY_IMPORTED`.
### Reguli Business
- **Persoana**: shipping name = persoana pe eticheta = beneficiarul facturii
- **Adresa**: cand billing ≠ shipping → adresa shipping pentru ambele (facturare + livrare)
@@ -182,6 +194,87 @@ gomag-vending/
---
## Facturi & Cache
Facturile sunt verificate live din Oracle si cacate in SQLite (`factura_*` pe tabelul `orders`).
### Sursa Oracle
```sql
SELECT id_comanda, numar_act, serie_act,
total_fara_tva, total_tva, total_cu_tva,
TO_CHAR(data_act, 'YYYY-MM-DD')
FROM vanzari
WHERE id_comanda IN (...) AND sters = 0
```
### Populare Cache
1. **Dashboard** (`GET /api/dashboard/orders`) — comenzile fara cache sunt verificate live si cacate automat la fiecare request
2. **Detaliu comanda** (`GET /api/sync/order/{order_number}`) — verifica Oracle live daca nu e caat
3. **Refresh manual** (`POST /api/dashboard/refresh-invoices`) — refresh complet pentru toate comenzile
### Refresh Complet — `/api/dashboard/refresh-invoices`
Face trei verificari in Oracle si actualizeaza SQLite:
| Verificare | Actiune |
|------------|---------|
| Comenzi necacturate → au primit factura? | Cacheaza datele facturii |
| Comenzi cacturate → factura a fost stearsa? | Sterge cache factura |
| Toate comenzile importate → comanda stearsa din ROA? | Seteaza status `DELETED_IN_ROA` |
Returneaza: `{ checked, invoices_added, invoices_cleared, orders_deleted }`
---
## API Reference — Sync & Comenzi
### Sync
| Method | Path | Descriere |
|--------|------|-----------|
| POST | `/api/sync/start` | Porneste sync in background |
| POST | `/api/sync/stop` | Trimite semnal de stop |
| GET | `/api/sync/status` | Status curent + progres + last_run |
| GET | `/api/sync/history` | Istoric run-uri (paginat) |
| GET | `/api/sync/run/{id}` | Detalii run specific |
| GET | `/api/sync/run/{id}/log` | Log per comanda (JSON) |
| GET | `/api/sync/run/{id}/text-log` | Log text (live din memorie sau reconstruit din SQLite) |
| GET | `/api/sync/run/{id}/orders` | Comenzi run filtrate/paginate |
| GET | `/api/sync/order/{number}` | Detaliu comanda + items + ARTICOLE_TERTI + factura |
### Dashboard Comenzi
| Method | Path | Descriere |
|--------|------|-----------|
| GET | `/api/dashboard/orders` | Comenzi cu enrichment factura |
| POST | `/api/dashboard/refresh-invoices` | Force-refresh stare facturi + deleted orders |
**Parametri `/api/dashboard/orders`:**
- `period_days`: 3/7/30/90 sau 0 (toate sau interval custom)
- `period_start`, `period_end`: interval custom (cand `period_days=0`)
- `status`: `all` / `IMPORTED` / `SKIPPED` / `ERROR` / `UNINVOICED` / `INVOICED`
- `search`, `sort_by`, `sort_dir`, `page`, `per_page`
Filtrele `UNINVOICED` si `INVOICED` fac fetch din toate comenzile IMPORTED si filtreaza server-side dupa prezenta/absenta cache-ului de factura.
### Scheduler
| Method | Path | Descriere |
|--------|------|-----------|
| PUT | `/api/sync/schedule` | Configureaza (enabled, interval_minutes: 5/10/30) |
| GET | `/api/sync/schedule` | Status curent |
Configuratia este persistata in SQLite (`scheduler_config`).
### Settings
| Method | Path | Descriere |
|--------|------|-----------|
| GET | `/api/settings` | Citeste setari aplicatie |
| PUT | `/api/settings` | Salveaza setari |
| GET | `/api/settings/sectii` | Lista sectii Oracle |
| GET | `/api/settings/politici` | Lista politici preturi Oracle |
**Setari disponibile:** `transport_codmat`, `transport_vat`, `discount_codmat`, `discount_vat`, `transport_id_pol`, `discount_id_pol`, `id_pol`, `id_sectie`, `gomag_api_key`, `gomag_api_shop`, `gomag_order_days_back`, `gomag_limit`
---
## Status Implementare
| Faza | Status | Descriere |
@@ -204,13 +297,20 @@ gomag-vending/
Scriptul `deploy.ps1` face automat: git clone, venv, dependinte, detectare Oracle, `start.bat`, serviciu NSSM, configurare IIS reverse proxy.
### Update
### Update cod (pull + restart)
```powershell
# Ca Administrator
.\update.ps1
```
Sau manual:
```powershell
cd C:\gomag-vending
git pull origin main
nssm restart GoMagVending
```
### Configurare `.env` pe Windows
```ini
@@ -220,24 +320,39 @@ ORACLE_PASSWORD=****
ORACLE_DSN=ROA
TNS_ADMIN=C:\roa\instantclient_11_2_0_2
INSTANTCLIENTPATH=C:\app\Server\product\18.0.0\dbhomeXE\bin
SQLITE_DB_PATH=api/data/import.db
JSON_OUTPUT_DIR=api/data/orders
APP_PORT=5003
ID_POL=39
ID_GESTIUNE=0
ID_SECTIE=6
GOMAG_API_KEY=...
GOMAG_API_SHOP=...
GOMAG_ORDER_DAYS_BACK=7
GOMAG_LIMIT=100
```
**Important:**
- `TNS_ADMIN` = folderul care contine `tnsnames.ora` (NU fisierul in sine)
- `ORACLE_DSN` = alias-ul exact din `tnsnames.ora`
- `INSTANTCLIENTPATH` = calea catre Oracle bin (thick mode)
- `INSTANTCLIENTPATH` = calea catre Oracle bin (thick mode, Oracle 10g/11g)
- `FORCE_THIN_MODE=true` = elimina necesitatea Instant Client (Oracle 12.1+)
- Setarile din `.env` pot fi suprascrise din UI → `Setari` → salvate in SQLite
### Serviciu Windows (NSSM)
```powershell
nssm restart GoMagVending # restart
nssm status GoMagVending # status
nssm stop GoMagVending # stop
nssm restart GoMagVending # restart serviciu
nssm status GoMagVending # status serviciu
nssm stop GoMagVending # stop serviciu
nssm start GoMagVending # start serviciu
```
Loguri serviciu: `logs/service_stdout.log`, `logs/service_stderr.log`
Loguri aplicatie: `logs/sync_comenzi_*.log`
**Nota:** Userul `gomag` nu are drepturi de admin — `nssm restart` necesita PowerShell Administrator direct pe server.
### Depanare SSH
```bash
@@ -258,9 +373,14 @@ Get-Process *python* | Select-Object Id,ProcessName,Path
# Verificare loguri recente
Get-ChildItem C:\gomag-vending\logs\*.log | Sort-Object LastWriteTime -Descending | Select-Object -First 3
```
**Nota SSH:** Userul `gomag` nu are drepturi de admin — `nssm restart` si `net stop/start` necesita PowerShell Administrator direct pe server.
# Test sync manual (verifica ca Oracle pool porneste)
curl http://localhost:5003/health
curl -X POST http://localhost:5003/api/sync/start
# Refresh facturi manual
curl -X POST http://localhost:5003/api/dashboard/refresh-invoices
```
### Probleme frecvente
@@ -269,6 +389,9 @@ Get-ChildItem C:\gomag-vending\logs\*.log | Sort-Object LastWriteTime -Descendin
| `ORA-12154: TNS:could not resolve` | `TNS_ADMIN` gresit sau `tnsnames.ora` nu contine alias-ul DSN | Verifica `TNS_ADMIN` in `.env` + alias in `tnsnames.ora` |
| `ORA-04088: LOGON_AUDIT_TRIGGER` + `Nu aveti licenta pentru PYTHON` | Trigger ROA blocheaza executabile nelicențiate | Adauga `python.exe` (calea completa) in ROASUPORT |
| `503 Service Unavailable` pe `/api/articles/search` | Oracle pool nu s-a initializat | Verifica logul `sync_comenzi_*.log` pentru eroarea exacta |
| Facturile nu apar in dashboard | Cache SQLite gol — invoice_service nu a putut interoga Oracle | Apasa butonul Refresh Facturi din dashboard sau `POST /api/dashboard/refresh-invoices` |
| Comanda apare ca `DELETED_IN_ROA` | Comanda a fost stearsa manual din ROA | Normal — marcat automat la refresh |
| Scheduler nu porneste dupa restart | Config pierduta | Verifica SQLite `scheduler_config` sau reconfigureaza din UI |
---

View File

@@ -492,18 +492,26 @@ async def dashboard_orders(page: int = 1, per_page: int = 50,
@router.post("/api/dashboard/refresh-invoices")
async def refresh_invoices():
"""Force-refresh invoice status from Oracle for all uninvoiced imported orders."""
try:
uninvoiced = await sqlite_service.get_uninvoiced_imported_orders()
if not uninvoiced:
return {"updated": 0, "message": "Nicio comanda de verificat"}
"""Force-refresh invoice/order status from Oracle.
Checks:
1. Uninvoiced orders → did they get invoiced?
2. Invoiced orders → was the invoice deleted?
3. All imported orders → was the order deleted from ROA?
"""
try:
invoices_added = 0
invoices_cleared = 0
orders_deleted = 0
# 1. Check uninvoiced → new invoices
uninvoiced = await sqlite_service.get_uninvoiced_imported_orders()
if uninvoiced:
id_comanda_list = [o["id_comanda"] for o in uninvoiced]
invoice_data = await asyncio.to_thread(
invoice_service.check_invoices_for_orders, id_comanda_list
)
id_to_order = {o["id_comanda"]: o["order_number"] for o in uninvoiced}
updated = 0
for idc, inv in invoice_data.items():
order_num = id_to_order.get(idc)
if order_num and inv.get("facturat"):
@@ -516,10 +524,41 @@ async def refresh_invoices():
total_cu_tva=inv.get("total_cu_tva"),
data_act=inv.get("data_act"),
)
updated += 1
return {"updated": updated, "checked": len(uninvoiced)}
invoices_added += 1
# 2. Check invoiced → deleted invoices
invoiced = await sqlite_service.get_invoiced_imported_orders()
if invoiced:
id_comanda_list = [o["id_comanda"] for o in invoiced]
invoice_data = await asyncio.to_thread(
invoice_service.check_invoices_for_orders, id_comanda_list
)
for o in invoiced:
if o["id_comanda"] not in invoice_data:
await sqlite_service.clear_order_invoice(o["order_number"])
invoices_cleared += 1
# 3. Check all imported → deleted orders in ROA
all_imported = await sqlite_service.get_all_imported_orders()
if all_imported:
id_comanda_list = [o["id_comanda"] for o in all_imported]
existing_ids = await asyncio.to_thread(
invoice_service.check_orders_exist, id_comanda_list
)
for o in all_imported:
if o["id_comanda"] not in existing_ids:
await sqlite_service.mark_order_deleted_in_roa(o["order_number"])
orders_deleted += 1
checked = len(uninvoiced) + len(invoiced) + len(all_imported)
return {
"checked": checked,
"invoices_added": invoices_added,
"invoices_cleared": invoices_cleared,
"orders_deleted": orders_deleted,
}
except Exception as e:
return {"error": str(e), "updated": 0}
return {"error": str(e), "invoices_added": 0}
@router.put("/api/sync/schedule")

View File

@@ -43,3 +43,33 @@ def check_invoices_for_orders(id_comanda_list: list) -> dict:
database.pool.release(conn)
return result
def check_orders_exist(id_comanda_list: list) -> set:
"""Check which id_comanda values still exist in Oracle COMENZI (sters=0).
Returns set of id_comanda that exist.
"""
if not id_comanda_list or database.pool is None:
return set()
existing = set()
conn = database.get_oracle_connection()
try:
with conn.cursor() as cur:
for i in range(0, len(id_comanda_list), 500):
batch = id_comanda_list[i:i+500]
placeholders = ",".join([f":c{j}" for j in range(len(batch))])
params = {f"c{j}": cid for j, cid in enumerate(batch)}
cur.execute(f"""
SELECT id_comanda FROM COMENZI
WHERE id_comanda IN ({placeholders}) AND sters = 0
""", params)
for row in cur:
existing.add(row[0])
except Exception as e:
logger.warning(f"Order existence check failed: {e}")
finally:
database.pool.release(conn)
return existing

View File

@@ -781,6 +781,83 @@ async def update_order_invoice(order_number: str, serie: str = None,
await db.close()
async def get_invoiced_imported_orders() -> list:
"""Get imported orders that HAVE cached invoice data (for re-verification)."""
db = await get_sqlite()
try:
cursor = await db.execute("""
SELECT order_number, id_comanda FROM orders
WHERE status IN ('IMPORTED', 'ALREADY_IMPORTED')
AND id_comanda IS NOT NULL
AND factura_numar IS NOT NULL AND factura_numar != ''
""")
rows = await cursor.fetchall()
return [dict(r) for r in rows]
finally:
await db.close()
async def get_all_imported_orders() -> list:
"""Get ALL imported orders with id_comanda (for checking if deleted in ROA)."""
db = await get_sqlite()
try:
cursor = await db.execute("""
SELECT order_number, id_comanda FROM orders
WHERE status IN ('IMPORTED', 'ALREADY_IMPORTED')
AND id_comanda IS NOT NULL
""")
rows = await cursor.fetchall()
return [dict(r) for r in rows]
finally:
await db.close()
async def clear_order_invoice(order_number: str):
"""Clear cached invoice data when invoice was deleted in ROA."""
db = await get_sqlite()
try:
await db.execute("""
UPDATE orders SET
factura_serie = NULL,
factura_numar = NULL,
factura_total_fara_tva = NULL,
factura_total_tva = NULL,
factura_total_cu_tva = NULL,
factura_data = NULL,
invoice_checked_at = datetime('now'),
updated_at = datetime('now')
WHERE order_number = ?
""", (order_number,))
await db.commit()
finally:
await db.close()
async def mark_order_deleted_in_roa(order_number: str):
"""Mark an order as deleted in ROA — clears id_comanda and invoice cache."""
db = await get_sqlite()
try:
await db.execute("""
UPDATE orders SET
status = 'DELETED_IN_ROA',
id_comanda = NULL,
id_partener = NULL,
factura_serie = NULL,
factura_numar = NULL,
factura_total_fara_tva = NULL,
factura_total_tva = NULL,
factura_total_cu_tva = NULL,
factura_data = NULL,
invoice_checked_at = NULL,
error_message = 'Comanda stearsa din ROA',
updated_at = datetime('now')
WHERE order_number = ?
""", (order_number,))
await db.commit()
finally:
await db.close()
# ── App Settings ─────────────────────────────────
async def get_app_settings() -> dict:

View File

@@ -463,17 +463,19 @@ async def run_sync(id_pol: int = None, id_sectie: int = None, run_id: str = None
logger.warning("Too many errors, stopping sync")
break
# Step 4b: Invoice check — update cached invoice data
_update_progress("invoices", "Checking invoices...", 0, 0)
# Step 4b: Invoice & order status check — sync with Oracle
_update_progress("invoices", "Checking invoices & order status...", 0, 0)
invoices_updated = 0
invoices_cleared = 0
orders_deleted = 0
try:
# 4b-1: Uninvoiced → check for new invoices
uninvoiced = await sqlite_service.get_uninvoiced_imported_orders()
if uninvoiced:
id_comanda_list = [o["id_comanda"] for o in uninvoiced]
invoice_data = await asyncio.to_thread(
invoice_service.check_invoices_for_orders, id_comanda_list
)
# Build reverse map: id_comanda → order_number
id_to_order = {o["id_comanda"]: o["order_number"] for o in uninvoiced}
for idc, inv in invoice_data.items():
order_num = id_to_order.get(idc)
@@ -488,10 +490,39 @@ async def run_sync(id_pol: int = None, id_sectie: int = None, run_id: str = None
data_act=inv.get("data_act"),
)
invoices_updated += 1
# 4b-2: Invoiced → check for deleted invoices
invoiced = await sqlite_service.get_invoiced_imported_orders()
if invoiced:
id_comanda_list = [o["id_comanda"] for o in invoiced]
invoice_data = await asyncio.to_thread(
invoice_service.check_invoices_for_orders, id_comanda_list
)
for o in invoiced:
if o["id_comanda"] not in invoice_data:
await sqlite_service.clear_order_invoice(o["order_number"])
invoices_cleared += 1
# 4b-3: All imported → check for deleted orders in ROA
all_imported = await sqlite_service.get_all_imported_orders()
if all_imported:
id_comanda_list = [o["id_comanda"] for o in all_imported]
existing_ids = await asyncio.to_thread(
invoice_service.check_orders_exist, id_comanda_list
)
for o in all_imported:
if o["id_comanda"] not in existing_ids:
await sqlite_service.mark_order_deleted_in_roa(o["order_number"])
orders_deleted += 1
if invoices_updated:
_log_line(run_id, f"Facturi actualizate: {invoices_updated} comenzi facturate")
_log_line(run_id, f"Facturi noi: {invoices_updated} comenzi facturate")
if invoices_cleared:
_log_line(run_id, f"Facturi sterse: {invoices_cleared} facturi eliminate din cache")
if orders_deleted:
_log_line(run_id, f"Comenzi sterse din ROA: {orders_deleted}")
except Exception as e:
logger.warning(f"Invoice check failed: {e}")
logger.warning(f"Invoice/order status check failed: {e}")
# Step 5: Update sync run
total_imported = imported_count + already_imported_count # backward-compat
@@ -514,6 +545,8 @@ async def run_sync(id_pol: int = None, id_sectie: int = None, run_id: str = None
"errors": error_count,
"missing_skus": len(validation["missing"]),
"invoices_updated": invoices_updated,
"invoices_cleared": invoices_cleared,
"orders_deleted_in_roa": orders_deleted,
}
_update_progress("completed",

View File

@@ -437,6 +437,7 @@ function orderStatusBadge(status) {
case 'ALREADY_IMPORTED': return '<span class="badge bg-info">Deja importat</span>';
case 'SKIPPED': return '<span class="badge bg-warning">Omis</span>';
case 'ERROR': return '<span class="badge bg-danger">Eroare</span>';
case 'DELETED_IN_ROA': return '<span class="badge bg-dark">Sters din ROA</span>';
default: return `<span class="badge bg-secondary">${esc(status)}</span>`;
}
}

View File

@@ -38,6 +38,7 @@ function orderStatusBadge(status) {
case 'ALREADY_IMPORTED': return '<span class="badge bg-info">Deja importat</span>';
case 'SKIPPED': return '<span class="badge bg-warning">Omis</span>';
case 'ERROR': return '<span class="badge bg-danger">Eroare</span>';
case 'DELETED_IN_ROA': return '<span class="badge bg-dark">Sters din ROA</span>';
default: return `<span class="badge bg-secondary">${esc(status)}</span>`;
}
}