sync_service gains DATA_ERRORS tuple + two new primitives:
_record_phase_err(run_id, phase, err)
Logs, appends to run text log, persists to sync_phase_failures.
_check_escalation()
Reads the last 3 runs and returns the first phase that has failed
all 3 in a row, or (None, counts) otherwise.
run_sync now runs a pre-flight escalation check — if a phase has failed
3 consecutive runs, the incoming sync is halted with
status='halted_escalation' and a descriptive error_message. The
dashboard Start Sync button can still override (UI comes in the next
PR2 phase).
Wrapped phases (DATA_ERRORS caught, sync continues):
cancelled_batch, already_batch, addresses_batch, skipped_batch,
price_sync, invoice_check, anaf_backfill.
Partner mismatch retains its existing per-order guards. OperationalError
and OS-level errors still propagate to the top-level handler (halt).
6 unit tests cover record + counts + threshold + mixed-phase +
short-circuit + DATA_ERRORS contract. Full CI green: 251 unit + 33 e2e.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SKUs mapped externally (via SSH script or direct SQL) never triggered
resolve_missing_sku(), leaving them stuck as unresolved=0 indefinitely.
New reconcile_unresolved_missing_skus() revalidates ALL unresolved SKUs
against Oracle at sync, rescan, and CSV import time. Fail-soft on Oracle
down. Clears the 7 prod false positives on next sync or manual rescan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. SQLite order_items overwrite on re-import (VELA CAFE #484669620):
add_order_items, save_orders_batch, mark_order_deleted_in_roa now use
DELETE + INSERT so GoMag quantity changes propagate to dashboard.
2. PL/SQL strict CUI lookup tolerates whitespace (FG COFFE #485065210):
cauta_partener_dupa_cod_fiscal regex ^RO\d → ^RO\s*\d; IN-set uses
canonical v_ro_cui. Platitor/neplatitor business rule preserved.
Python defensive: re.sub whitespace collapse in determine_partner_data.
3. New PJ partners use ANAF official denumire (denumire_override) instead
of GoMag company_name. Existing partners (found by CUI) untouched.
Tests: 18 new (5 SQLite unit, 8 Python unit, 5 Oracle PL/SQL). All green
locally: 228 unit + 26 oracle + 33 e2e.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes false negatives where city spellings differ slightly (e.g.
"Sfântu Ilie" vs "SFINTU ILIE") or ROA stores "BUCURESTI SECTORUL 1"
while GoMag sends "Municipiul București". Both backend (_addr_match)
and frontend (addrMatch) now use identical SOUNDEX logic mirroring
Oracle's implementation.
Also fixes field order: etaj before apart in r_street concatenation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Oracle parser failed to extract sc/ap/et when GoMag addresses had no
commas. Added REGEXP_REPLACE to insert commas before address keywords
in v_strada before the comma-split, ensuring the token parser always
fires. Also added 5 Oracle integration tests calling
parseaza_adresa_semicolon directly, and improved diacritics handling
in addr_match (Python + JS).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes caused partner_mismatch=1 to loop indefinitely:
1. No-CUI company orders (is_pj=1, no cod_fiscal): old code flagged as
mismatch every cycle. Fixed by requiring new_cf to be non-null for
PF→PJ detection. Stale flags from old code cleared via new
clear_stale_partner_mismatches_no_cui() for out-of-window orders.
2. same_partner resync path did not update cod_fiscal_gomag in SQLite.
On next cycle GoMag returned a CUI but stored_cf was still NULL →
re-detected as mismatch. Fixed by also calling update_partner_resync_data
(not just update_partner_mismatch_batch) in the same_partner branch.
3. GoMag sends CUI with space: 'RO 17922480'. The _strip_ro() regex
^RO left the space → ' 17922480' != '17922480' → false mismatch.
Fixed: changed regex to ^RO\s* and added .strip().
Also adds diagnostic logger.info lines for mismatch detection/resync counts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GoMag sends baseprice (catalog price) alongside price (discounted).
When baseprice > price, the item is volume-discounted — skip ROA
price comparison and show amber "Disc." badge instead of false
mismatch. Strikethrough baseprice in price column for transparency.
Pipeline: parse baseprice → store in SQLite → skip in validation →
pass flag to frontend → render badge (desktop + mobile pill badge
with aria-label, opacity 0.6 for dark mode).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ANAF notFound items are plain integers, not dicts — caused 'int has no
attribute get'. 4xx errors (like 404) no longer retry uselessly. ANAF
errors now appear in the UI sync log via log_fn callback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Orders synced before address_mismatch was deployed had stale 0 values,
causing missing blue dots in the dashboard. Adds startup backfill from
stored address JSON + recomputes on each sync for ALREADY_IMPORTED orders.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove TIER 2 address lookup (county+city without street) from PL/SQL — creates
new address when street differs instead of reusing wrong one
- Replace generic "N diferente" badge with typed micro-badges (CUI, Denumire, TVA,
Adr. livr., Adr. fact., Preturi) with red/amber semantic colors
- Extend addrMatch() regex to strip full Romanian address words (STRADA, NUMAR, BLOC,
COMUNA, SAT, MUNICIPIUL, etc.) — fixes "Strada X" vs "X" false positives
- Extend normalize_company_name() for II, PFA, INTREPRINDERE INDIVIDUALA legal forms
- Persist address_mismatch to SQLite so "Dif." filter includes address-only diffs
- Add red/amber indicator dots to desktop table and mobile list rows
- 12 unit tests for normalization and server-side address matching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Customers often swap firstname/lastname in GoMag forms, causing duplicate
partner creation in Oracle. Fix with two layers:
- Python: sort PF name words alphabetically before Oracle lookup
- PL/SQL: add Step 2b permutation search (2-3 word names, PF only)
- Normalize name order to lastname+firstname across all Python files
- Add diagnostic SQL for finding existing reversed-name duplicates
- Add Oracle integration test for reverse-name matching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Romanian CUI check digit algorithm (key 753217532) validates CUIs
before ANAF lookup. New sanitize_cui() fixes OCR typos (O→0, I→1)
and verifies checksum, logging warnings for invalid CUIs.
Applied at both ANAF batch verification and per-order import steps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Orders imported before the ANAF dedup feature had no anaf_platitor_tva.
Step 4c now auto-backfills on each sync: batch cache lookup, single
ANAF API call for uncached CUIs, bulk DB update in one transaction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PL/SQL cauta_partener_dupa_cod_fiscal gains p_strict_search param:
- strict (=1): search only exact CUI form (ANAF-determined)
- dual (NULL): search all forms (existing anti-dedup behavior)
Skip denomination fallback when strict to force new partner creation.
Python: country filter excludes foreign companies from ANAF batch,
anaf_strict flag threaded sync→import→PL/SQL, normalize RO-space
in cod_fiscal_adjusted comparison to eliminate false positives.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix ANAF API: extract CUI from date_generale (not top-level), fix
notFound casing (capital F)
- Fix missing facturare address when same ID as livrare (copy instead
of skip)
- Replace ANAF cache pre-population stub with real logic (3-month CUIs)
- Restructure order detail modal: inline 2-col GOMAG|ROA layout with
compact address lines replacing collapsed sections
- Fix addrMatch() to use field-level comparison with Romanian
abbreviation stripping (STR, NR, BL, SC, AP, ET, ETAJ, APART)
- Add dashboard "Diferente" filter pill for ANAF-adjusted orders
- Update e2e test for new modal structure
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevent partner duplicates via ANAF CUI verification and dual PL/SQL
search. Fix address matching with street-level comparison and diacritics
normalization. Show partner/address comparison in order detail modal.
- New anaf_service.py: batch ANAF API client with chunking, retry, cache
- PL/SQL: dual CUI search (bare/RO+bare/RO space+bare), 3-tier address
search (street+city+id_loc → city+id_loc → create), strip_diacritics
at storage for addresses and partner names
- SQLite: anaf_cache table, 12 new order columns for partner/address data
- import_service: cod_fiscal_override param, return partner/address from Oracle
- sync_service: ANAF batch integration, denomination mismatch detection,
cache pre-population trigger
- Router: enriched order_detail with partner_info + addresses JSON
- UI: collapsible Detalii Partener + Adrese Comparativ sections in modal,
auto-expand on mismatch, ANAF badges, mobile address cards
- Dashboard: address quality attention indicator
- New scan_duplicate_partners.py script for one-time duplicate audit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Existing 741 rows also updated via UPPER() on customer_name,
shipping_name, billing_name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix vat_included comparison: GoMag API returns int 1, not str "1",
causing all prices to be multiplied by TVA again (double TVA)
- Normalize vat_included to string in gomag_client at parse time
- Price sync now processes kit components individually by looking up
each component's CODMAT as standalone GoMag product
- Add _insert_component_price for components without existing Oracle price
- resolve_mapped_codmats: ROW_NUMBER dedup for CODMATs with multiple
NOM_ARTICOLE entries, prefer article with current stock
- pack_import_comenzi: merge_or_insert_articol to merge quantities when
same article appears from kit + individual on same order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
codmat_policy_map had CODMAT keys only, but build_articles_json looks
up by GoMag SKU — mapped articles like FRSETP250 never got per-article
id_pol, causing Oracle to use default sales policy and fail when price
exists only in production policy.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Oracle PL/SQL: kit pricing logic with Mode A (distributed discount) and
Mode B (separate discount line), dual policy support, PRETURI_CU_TVA flag
- Eliminate procent_pret from entire stack (Oracle, Python, JS, HTML)
- New settings: kit_pricing_mode, kit_discount_codmat, price_sync_enabled
- Settings UI: cards for Kit Pricing and Price Sync configuration
- Mappings UI: kit badges with lazy-loaded component prices from price list
- Price sync from orders: auto-update ROA prices when web prices differ
- Catalog price sync: new service to sync all GoMag product prices to ROA
- Kit component price validation: pre-check prices before import
- New endpoint GET /api/mappings/{sku}/prices for component price display
- New endpoints POST /api/price-sync/start, GET status, GET history
- DDL script 07_drop_procent_pret.sql (run after deploy confirmation)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace single-select gestiune dropdown with multi-select checkboxes.
Settings stores comma-separated IDs, Python builds IN clause with bind
variables, Oracle PL/SQL splits CSV via REGEXP_SUBSTR for stock lookup.
Empty selection = all warehouses (unchanged behavior).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add production pricing policy (id_pol_productie) for articles with cont 341/345,
smart discount VAT splitting across multiple rates, per-article id_pol support,
and mapped SKU price validation. Settings UI updated with new controls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sync card was showing previous run data after sync completed because the
last_run query excluded the current run_id even after it finished. Now only
excludes during active running state.
All datetime.now() and SQLite datetime('now') replaced with Europe/Bucharest
timezone to fix times displayed 2 hours behind (was using UTC).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unified id_articol selection logic in Python (resolve_codmat_ids) and PL/SQL
(resolve_id_articol): filters sters=0 AND inactiv=0, prefers article with
stock in configured gestiune, falls back to MAX(id_articol). Eliminates
mismatch where Python and PL/SQL could pick different id_articol for the
same CODMAT, causing ORA-20000 price-not-found errors.
- Add resolve_codmat_ids helper in validation_service.py (single batch query)
- Refactor validate_skus/validate_prices/ensure_prices to use it
- Add resolve_id_articol function in PL/SQL package body
- Add p_id_gestiune parameter to importa_comanda (spec + body)
- Add /api/settings/gestiuni endpoint and id_gestiune setting
- Add gestiune dropdown in settings UI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add web_status column to orders table (generic name for platform status)
- Filter cancelled orders during sync, record as CANCELLED in SQLite
- Soft-delete previously-imported cancelled orders in Oracle (if not invoiced)
- Add CANCELLED filter pill + badge in dashboard UI
- New soft_delete_order_in_roa() and mark_order_cancelled() functions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When billing is on a company, customer_name now uses billing.company_name
instead of shipping person name. This aligns SQLite customer_name with the
partner created in ROA by import_service, making order-invoice correlation
possible in the dashboard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, orders deleted from Oracle (sters=1) remained as IMPORTED
in SQLite, and deleted invoices kept stale cache data. Now the refresh
button and sync cycle re-verify all imported orders against Oracle:
- Deleted orders → marked DELETED_IN_ROA with cleared id_comanda
- Deleted invoices → invoice cache fields cleared
- New status badge for DELETED_IN_ROA in dashboard and logs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PL/SQL: handle duplicate CODMAT in nom_articole with MAX(id_articol)
- import_service: add explicit conn.rollback() on Oracle errors
- sync_service: auto-fix stale ERROR orders that exist in Oracle
- invoice_service: add data_act (invoice date) from vanzari table
- sync router: new POST /api/dashboard/refresh-invoices endpoint
- order detail: enrich with invoice data (serie, numar, data factura)
- dashboard: refresh invoices button (desktop + mobile icon)
- quick map modal: compact single-row layout, pre-populate existing mappings
- quick map: link on SKU column instead of CODMAT
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parse delivery.total and discounts[] from GoMag JSON into new
delivery_cost/discount_total fields. Add app_settings table for
configuring transport/discount CODMAT codes. When configured,
transport and discount are appended as extra articles in the
Oracle import JSON. Reorder Total column in dashboard/logs tables
and show transport/discount breakdown in order detail modals.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parse order total from GoMag JSON, store in SQLite orders table,
and expose via sync run API. Enables total display in mobile flat rows.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New gomag_client.py service: async httpx client that downloads orders
from GoMag API with full pagination and 1s rate-limit sleep
- config.py: add GOMAG_API_KEY, GOMAG_API_SHOP, GOMAG_ORDER_DAYS_BACK,
GOMAG_LIMIT, GOMAG_API_URL settings
- sync_service.py: Phase 0 downloads fresh orders before reading JSONs;
graceful skip if API keys not configured
- start.sh: auto-detect INSTANTCLIENTPATH from .env, fallback to thin mode
- .env.example: document GoMag API variables
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Track already_imported/new_imported counts separately in sync_runs
and surface them in status API + dashboard last-run card
- Cache invoice data in SQLite orders table (factura_* columns);
dashboard falls back to Oracle only for uncached imported orders
- Resolve JSON_OUTPUT_DIR and SQLITE_DB_PATH relative to known
anchored roots in config.py, independent of CWD (fixes WSL2 start)
- Use single Oracle connection for entire validation phase (perf)
- Batch upsert web_products instead of one-by-one
- Remove stale VFP scripts (replaced by gomag-vending.prg workflow)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace import_orders (insert-per-run) with orders table (one row per
order, upsert on conflict). Eliminates dedup CTE on every dashboard
query and prevents unbounded row growth at 4-500 orders/sync.
Key changes:
- orders table: PK order_number, upsert via ON CONFLICT DO UPDATE;
COALESCE preserves id_comanda once set; times_skipped auto-increments
- sync_run_orders: lightweight junction (sync_run_id, order_number)
replaces sync_run_id column on orders
- order_items: PK changed to (order_number, sku), INSERT OR IGNORE
- Auto-migration in init_sqlite(): import_orders → orders on first boot,
old table renamed to import_orders_bak
- /api/dashboard/orders: period_days param (3/7/30/0=all, default 7)
- Dashboard: period selector buttons in orders card header
- start.sh: stop existing process on port 5003 before restart;
remove --reload (broken on WSL2 /mnt/e/)
- Add invoice_service, E2E Playwright tests, Oracle package updates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add SSE event bus in sync_service (subscribe/unsubscribe/_emit)
- Add GET /api/sync/stream SSE endpoint for real-time sync progress
- Rewrite logs.html: unified runs table + live feed + summary + filters
- Rewrite logs.js: SSE EventSource client, run selection, pagination
- Dashboard: clickable runs navigate to /logs?run=, sync started banner
- Remove "Import Comenzi" nav item, delete sync_detail.html
- Add error_message column to sync_runs table with migration
- Fix: export TNS_ADMIN as OS env var so oracledb finds tnsnames.ora
- Fix: use get_oracle_connection() instead of direct pool.acquire()
- Fix: CRM_POLITICI_PRET_ART INSERT to match actual table schema
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Flask admin with FastAPI app (api/app/) featuring:
- Dashboard with stat cards, sync control, and history
- Mappings CRUD for ARTICOLE_TERTI with CSV import/export
- Article autocomplete from NOM_ARTICOLE
- SKU pre-validation before import
- Sync orchestration: read JSONs -> validate -> import -> log to SQLite
- APScheduler for periodic sync from UI
- File logging to logs/sync_comenzi_YYYYMMDD_HHMMSS.log
- Oracle pool None guard (503 vs 500 on unavailable)
Test suite:
- test_app_basic.py: 30 tests (imports + routes) without Oracle
- test_integration.py: 9 integration tests with Oracle
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
