New table sync_phase_failures(run_id, phase, error_summary, created_at)
with index on (phase, created_at). Minimal schema — no raw payload, no
PII — stores just enough to answer "did phase X fail in the last N
runs?" for the escalation check and the /api/sync/health pill.
Helpers in sqlite_service:
record_phase_failure(run_id, phase, error_summary)
INSERT OR REPLACE semantics (one row per run+phase), then prunes
to the most recent 100 sync_runs. error_summary clipped at 500
chars defensively.
get_recent_phase_failures(limit=3) → {phase: count} across the last N
runs, ordered by started_at desc.
6 unit tests cover creation, counting, pruning, empty state,
idempotency, and limit semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_safe_upsert_order_items(db, order_number, items) wraps the
DELETE + INSERT OR REPLACE pair in SAVEPOINT items. On
IntegrityError / ValueError / TypeError it rolls the savepoint
back, tags the parent order MALFORMED, logs to the error history
file, and returns False to the caller. add_order_items now delegates
to this helper so a single bad payload cannot leave order_items in
a split state.
2 integration tests: happy path + simulated INSERT crash via
aiosqlite monkeypatch. Existing order_items overwrite regression
tests still pass (5/5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
save_orders_batch now runs in three tiers:
1. validate_structural pre-flight splits each payload into valid or
MALFORMED. MALFORMED rows persist with status + error_message + no
items, and an append-only entry lands in sync_errors_history.log.
2. Optimistic executemany over the valid list inside a SAVEPOINT batch.
3. On IntegrityError / ValueError / TypeError, rollback the savepoint
and fall back to per-order SAVEPOINT inserts so a single bad row
cannot poison the rest of the batch.
Mid-loop SAVEPOINT rollback failure now triggers _safe_reconnect:
commit whatever survived, close the broken connection, open a fresh
one and keep processing. Preserves MALFORMED rows recorded earlier —
addresses the outside-voice gap where a crashed connection would lose
uncommitted malformed evidence.
Adds OrderStatus.MALFORMED and helper functions:
_insert_orders_only — orders + sync_run_orders, no items
_insert_valid_batch — happy-path bulk executemany
_insert_single_order — per-order execute within savepoint
_mark_malformed — non-mutating copy with wiped items
_safe_reconnect — commit-close-reconnect guard
8 integration tests covering regression 485224762, structural
pre-flight, per-order isolation on runtime fail, caller-dict
immutability, and reconnect durability. 239 unit + 33 e2e green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_log_order_error_history(order_number, msg) writes to
logs/sync_errors_history.log via a dedicated RotatingFileHandler
(100MB × 12 backups). Logger is lazy-initialised and non-propagating
so it doesn't pollute the root logger.
Purpose: orders.error_message is overwritten when a retry succeeds,
so the history log preserves permanent audit of every malformed-order
event regardless of later outcome. Helper never raises — callers are
already in a degraded path.
3 unit tests: append semantics, multi-order, exception isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
validate_structural(order) runs before save_orders_batch insert.
Catches malformed payloads (MISSING_FIELD, INVALID_DATE, EMPTY_ITEMS,
INVALID_QUANTITY, INVALID_PRICE) that would otherwise crash the batch
insert or downstream pipeline. 17 unit tests cover each rule.
Does NOT validate SKU existence — redundant with _dedup_items_by_sku
pass-through and validate_skus Oracle lookup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Production sync was failing every minute with:
UNIQUE constraint failed: order_items.order_number, order_items.sku
GoMag occasionally returns the same SKU on multiple lines within one order
(configurable products, promo splits). The order_items PK is
(order_number, sku), so the raw batch insert violates UNIQUE and aborts
the entire sync — blocking partner-mismatch updates, address refresh,
and items repopulation for already-imported orders.
Added _dedup_items_by_sku() helper. Applied in save_orders_batch
(cancelled/already/skipped paths) and add_order_items (retry/sync import
paths). Keeps first price/vat/name, sums quantities on collision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Script failed with "SQLite not initialized" because module-level connection
state wasn't set up when invoked standalone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two production bugs from VENDING (order 485224762, 2026-04-22):
1. Oracle: ORA-20000 when a GoMag order contains a kit SKU whose expansion
includes CODMAT X plus a second item with SKU=X. Two article-insert
call-sites in PACK_IMPORT_COMENZI bypassed merge_or_insert_articol —
line 622 (NOM_ARTICOLE fallback) and line 538 (kit discount line).
Both now use merge_or_insert_articol for consistent dedup semantics.
Regression test added in test_complete_import.py covering the exact
kit-plus-direct scenario.
2. SQLite: retry_service._download_and_reimport refreshed orders row but
never repopulated order_items. Combined with mark_order_deleted_in_roa
(which wipes items), any retry/resync left the UI showing "Niciun
articol" despite successful Oracle import. Retry now rebuilds items
from the fresh GoMag download on both success and error paths,
mirroring sync_service.
Includes scripts/backfill_order_items.py — one-shot recovery for orders
already in this bad state. Reads settings, re-fetches from GoMag,
rewrites order_items without touching Oracle or order status.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In parseaza_adresa_semicolon, text după NR ("5 la non stop", "21 sat
Grozavesti corbii mari") era împins în p_numar și trunchiat brutal la
10 chars ("5 LA NON S", "21 SAT GRO").
Fix: când p_numar > 10 chars, prima componentă rămâne numar; restul se
clasifică:
- "SAT X ..." → p_localitate := "X ..." (satul = localitate, TIER
L1/L2/L3 existent rezolvă id_loc)
- "COM/ORAS/MUN X" → aruncat (deja în p_localitate din GoMag city)
- altceva (landmark ex "LA NON STOP") → concatenat în p_strada
Semnătura parseaza_adresa_semicolon neschimbată. Zero callers afectați.
Teste: landmark → strada, SAT → localitate, numar normal neschimbat.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SKUs mapped externally (via SSH script or direct SQL) never triggered
resolve_missing_sku(), leaving them stuck as unresolved=0 indefinitely.
New reconcile_unresolved_missing_skus() revalidates ALL unresolved SKUs
against Oracle at sync, rescan, and CSV import time. Fail-soft on Oracle
down. Clears the 7 prod false positives on next sync or manual rescan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. SQLite order_items overwrite on re-import (VELA CAFE #484669620):
add_order_items, save_orders_batch, mark_order_deleted_in_roa now use
DELETE + INSERT so GoMag quantity changes propagate to dashboard.
2. PL/SQL strict CUI lookup tolerates whitespace (FG COFFE #485065210):
cauta_partener_dupa_cod_fiscal regex ^RO\d → ^RO\s*\d; IN-set uses
canonical v_ro_cui. Platitor/neplatitor business rule preserved.
Python defensive: re.sub whitespace collapse in determine_partner_data.
3. New PJ partners use ANAF official denumire (denumire_override) instead
of GoMag company_name. Existing partners (found by CUI) untouched.
Tests: 18 new (5 SQLite unit, 8 Python unit, 5 Oracle PL/SQL). All green
locally: 228 unit + 26 oracle + 33 e2e.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cauta_partener_dupa_denumire nu filtra sters=0, deci importul PF
putea lega comanda la un partener sters=1 (bug GoMag #484668145 —
RADULESCU ANA MARIA, id_part=21946). Adaugat NVL(sters,0)=0 si
ORDER BY inactiv ASC pentru a prefera parteneri activi, in linie cu
cauta_partener_dupa_cod_fiscal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PJ: tooltip shows company on Facturat (display) vs shipping person.
PF ramburs: tooltip shows billing person vs shipping person when they
differ. Adds aria-label + title on indicator for keyboard/screen reader.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract shared setupAutocomplete() into shared.js so all three autocomplete
instances (mappings modal, inline add, quick-map modal) get keyboard nav
(ArrowDown/Up/Enter/Escape), scroll-safe blur handling, and capture-phase
keydown to prevent browser interception. Remove old onmousedown inline
handlers, use data-codmat/data-label attributes instead.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Kebab dropdown delete/resync used inlineConfirmAction which breaks inside
Bootstrap dropdowns (dropdown closes on click, hiding confirm state).
Replaced with confirm() dialog + direct async action with row feedback.
Detail modal resync/delete/retry now trigger onStatusChange callback to
refresh the orders table, so status dots update without page reload.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Bug 1: hover actions covered total column; replaced with kebab dropdown in dedicated 44px column
- Bug 2: resync/delete buttons kept stale state across modal opens; reset in modal init block
- Bug 3: delete success button was green (btn-success); changed to red (btn-danger)
- Dropdown styled per DESIGN.md: warm shadow, 8px radius, dark mode tokens
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Resync soft-deletes from Oracle then re-imports from GoMag with fresh
article data. Delete soft-deletes and marks DELETED_IN_ROA. Both have
invoice safety gates (refuse if invoiced or Oracle unavailable).
UI: split modal footer (Delete left, Resync+Close right), inline
confirm pattern (no native confirm()), dashboard row hover action
icons, disabled+tooltip for invoiced orders. 8 unit tests for safety
gates and happy paths.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes false negatives where city spellings differ slightly (e.g.
"Sfântu Ilie" vs "SFINTU ILIE") or ROA stores "BUCURESTI SECTORUL 1"
while GoMag sends "Municipiul București". Both backend (_addr_match)
and frontend (addrMatch) now use identical SOUNDEX logic mirroring
Oracle's implementation.
Also fixes field order: etaj before apart in r_street concatenation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Oracle parser failed to extract sc/ap/et when GoMag addresses had no
commas. Added REGEXP_REPLACE to insert commas before address keywords
in v_strada before the comma-split, ensuring the token parser always
fires. Also added 5 Oracle integration tests calling
parseaza_adresa_semicolon directly, and improved diacritics handling
in addr_match (Python + JS).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GoMag vs ROA price comparison generated too many false positives
(kits, volume discounts, special prices). Removes comparison columns,
dots, badges, catalog sync endpoints, and ~950 lines of dead code.
Keeps WRITE path (sync_prices_from_order) for kit pricing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes caused partner_mismatch=1 to loop indefinitely:
1. No-CUI company orders (is_pj=1, no cod_fiscal): old code flagged as
mismatch every cycle. Fixed by requiring new_cf to be non-null for
PF→PJ detection. Stale flags from old code cleared via new
clear_stale_partner_mismatches_no_cui() for out-of-window orders.
2. same_partner resync path did not update cod_fiscal_gomag in SQLite.
On next cycle GoMag returned a CUI but stored_cf was still NULL →
re-detected as mismatch. Fixed by also calling update_partner_resync_data
(not just update_partner_mismatch_batch) in the same_partner branch.
3. GoMag sends CUI with space: 'RO 17922480'. The _strip_ro() regex
^RO left the space → ' 17922480' != '17922480' → false mismatch.
Fixed: changed regex to ^RO\s* and added .strip().
Also adds diagnostic logger.info lines for mismatch detection/resync counts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README.md: replace old different_person logic with PJ/PF rule description
docs/adrese_facturare_variante.md: new file — decision rationale, implementation
summary, verification commands, history of the change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- diff dot for partner_mismatch uses --warning (orange) instead of --error
to distinguish from price mismatch (also red)
- modal ROA column shows "necunoscut - se va actualiza la urmatorul sync"
when denumire_roa is null but partner_mismatch=1 (orders imported before
the column existed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DPY-4000 (no tnsnames in dev) and ANAF 404/500 (mock server) are expected
in CI — add them to _KNOWN_ISSUES so the log monitor test passes at 100/100.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
baseprice > price was wrongly treated as "quantity discount" — it's just
GoMag's promotional price. Now: price_gomag <= pret_roa is always OK,
only flag when GoMag charges MORE than ROA. Reset cached price_match
at startup for re-evaluation. Fix dashboard dot color for mismatches.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TIER L2: SOUNDEX match pe judet (ex: CRAMPOIA→CRIMPOIA, varianta â/î).
TIER L3: pastreaza judetul corect rezolvat, nu mai reseteaza la default global.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GoMag sends baseprice (catalog price) alongside price (discounted).
When baseprice > price, the item is volume-discounted — skip ROA
price comparison and show amber "Disc." badge instead of false
mismatch. Strikethrough baseprice in price column for transparency.
Pipeline: parse baseprice → store in SQLite → skip in validation →
pass flag to frontend → render badge (desktop + mobile pill badge
with aria-label, opacity 0.6 for dark mode).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ANAF notFound items are plain integers, not dicts — caused 'int has no
attribute get'. 4xx errors (like 404) no longer retry uselessly. ANAF
errors now appear in the UI sync log via log_fn callback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TRANSLATE with UTF-8 literals was silently corrupted when compiled via
Windows sqlplus (ĂăÂâÎî→����, ȘșȚț→????). Replaced with REPLACE/UNISTR
for comma-below→cedilla normalization + CONVERT US7ASCII. Also applied
strip_diacritics to localitate/judet in TIER 1 lookup and locality search
(was only on strada), fixing 'FĂLTICENI' vs 'FALTICENI' BINARY mismatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Orders synced before address_mismatch was deployed had stale 0 values,
causing missing blue dots in the dashboard. Adds startup backfill from
stored address JSON + recomputes on each sync for ALREADY_IMPORTED orders.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CUI adjustment (RO prefix change) already implies TVA mismatch, so
suppress redundant TVA badge when CUI badge is shown. Remove
address_mismatch from DIFFS pill count since it's not an ANAF difference.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously the TVA badge/dot appeared whenever ANAF reported neplatitor.
Now it only appears when GoMag CUI prefix (RO=platitor) disagrees with
ANAF status — aligned across JS badges, red dots, SQL filter and count.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove TIER 2 address lookup (county+city without street) from PL/SQL — creates
new address when street differs instead of reusing wrong one
- Replace generic "N diferente" badge with typed micro-badges (CUI, Denumire, TVA,
Adr. livr., Adr. fact., Preturi) with red/amber semantic colors
- Extend addrMatch() regex to strip full Romanian address words (STRADA, NUMAR, BLOC,
COMUNA, SAT, MUNICIPIUL, etc.) — fixes "Strada X" vs "X" false positives
- Extend normalize_company_name() for II, PFA, INTREPRINDERE INDIVIDUALA legal forms
- Persist address_mismatch to SQLite so "Dif." filter includes address-only diffs
- Add red/amber indicator dots to desktop table and mobile list rows
- 12 unit tests for normalization and server-side address matching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Customers often swap firstname/lastname in GoMag forms, causing duplicate
partner creation in Oracle. Fix with two layers:
- Python: sort PF name words alphabetically before Oracle lookup
- PL/SQL: add Step 2b permutation search (2-3 word names, PF only)
- Normalize name order to lastname+firstname across all Python files
- Add diagnostic SQL for finding existing reversed-name duplicates
- Add Oracle integration test for reverse-name matching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove unreachable OCR-skip fallback (raw_bare can't be all-digits
if strip_ro_prefix changed it via OCR fix). Add real test for the
checksum result==10→0 branch using CUI 14186770.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers strip_ro_prefix (OCR fixes), validate_cui (format),
validate_cui_checksum (Romanian algorithm with key 753217532),
and sanitize_cui (end-to-end with warnings). Verified against
12 real CUIs from production orders.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Romanian CUI check digit algorithm (key 753217532) validates CUIs
before ANAF lookup. New sanitize_cui() fixes OCR typos (O→0, I→1)
and verifies checksum, logging warnings for invalid CUIs.
Applied at both ANAF batch verification and per-order import steps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
