Backend:
- GET /api/sync/health returns {last_sync_at, last_sync_status,
last_halt_reason, recent_phase_failures, escalation_phase, is_healthy}.
healthy when last run was completed (or none yet), no phase has
tripped the 3-in-a-row escalation, and recent failures <= 1.
- Dashboard + run-level endpoints include `malformed` count so the
Defecte pill can render.
Frontend:
- Health pill in .sync-card-controls with three states — healthy
(success green, check icon), warning (amber, triangle), escalated
(error red, x-octagon + glow). Tooltip exposes the halt reason and
the top phases with recent failures.
- Status-dot + badge add MALFORMED treatment via --compare orange,
distinct from ERROR red. DESIGN.md notes the diagnostic rationale
(ERROR = runtime, MALFORMED = payload source issue).
- Defecte filter pill on dashboard + logs pages. Mobile segmented
control includes Defecte count. Counts wired to the malformed key.
- startSync() shows a native confirm modal when state is
halted_escalation — operator override still possible, not silenced.
- ORDER_STATUS.MALFORMED mirror added to shared.js.
- Cache-bust: style.css v46, shared.js v47, dashboard.js v52,
logs.js v16.
5 endpoint tests cover empty state, completed, failed, escalated,
single-failure warning. Full CI: 257 unit + 33 e2e green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New table sync_phase_failures(run_id, phase, error_summary, created_at)
with index on (phase, created_at). Minimal schema — no raw payload, no
PII — stores just enough to answer "did phase X fail in the last N
runs?" for the escalation check and the /api/sync/health pill.
Helpers in sqlite_service:
record_phase_failure(run_id, phase, error_summary)
INSERT OR REPLACE semantics (one row per run+phase), then prunes
to the most recent 100 sync_runs. error_summary clipped at 500
chars defensively.
get_recent_phase_failures(limit=3) → {phase: count} across the last N
runs, ordered by started_at desc.
6 unit tests cover creation, counting, pruning, empty state,
idempotency, and limit semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_safe_upsert_order_items(db, order_number, items) wraps the
DELETE + INSERT OR REPLACE pair in SAVEPOINT items. On
IntegrityError / ValueError / TypeError it rolls the savepoint
back, tags the parent order MALFORMED, logs to the error history
file, and returns False to the caller. add_order_items now delegates
to this helper so a single bad payload cannot leave order_items in
a split state.
2 integration tests: happy path + simulated INSERT crash via
aiosqlite monkeypatch. Existing order_items overwrite regression
tests still pass (5/5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
save_orders_batch now runs in three tiers:
1. validate_structural pre-flight splits each payload into valid or
MALFORMED. MALFORMED rows persist with status + error_message + no
items, and an append-only entry lands in sync_errors_history.log.
2. Optimistic executemany over the valid list inside a SAVEPOINT batch.
3. On IntegrityError / ValueError / TypeError, rollback the savepoint
and fall back to per-order SAVEPOINT inserts so a single bad row
cannot poison the rest of the batch.
Mid-loop SAVEPOINT rollback failure now triggers _safe_reconnect:
commit whatever survived, close the broken connection, open a fresh
one and keep processing. Preserves MALFORMED rows recorded earlier —
addresses the outside-voice gap where a crashed connection would lose
uncommitted malformed evidence.
Adds OrderStatus.MALFORMED and helper functions:
_insert_orders_only — orders + sync_run_orders, no items
_insert_valid_batch — happy-path bulk executemany
_insert_single_order — per-order execute within savepoint
_mark_malformed — non-mutating copy with wiped items
_safe_reconnect — commit-close-reconnect guard
8 integration tests covering regression 485224762, structural
pre-flight, per-order isolation on runtime fail, caller-dict
immutability, and reconnect durability. 239 unit + 33 e2e green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_log_order_error_history(order_number, msg) writes to
logs/sync_errors_history.log via a dedicated RotatingFileHandler
(100MB × 12 backups). Logger is lazy-initialised and non-propagating
so it doesn't pollute the root logger.
Purpose: orders.error_message is overwritten when a retry succeeds,
so the history log preserves permanent audit of every malformed-order
event regardless of later outcome. Helper never raises — callers are
already in a degraded path.
3 unit tests: append semantics, multi-order, exception isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Production sync was failing every minute with:
UNIQUE constraint failed: order_items.order_number, order_items.sku
GoMag occasionally returns the same SKU on multiple lines within one order
(configurable products, promo splits). The order_items PK is
(order_number, sku), so the raw batch insert violates UNIQUE and aborts
the entire sync — blocking partner-mismatch updates, address refresh,
and items repopulation for already-imported orders.
Added _dedup_items_by_sku() helper. Applied in save_orders_batch
(cancelled/already/skipped paths) and add_order_items (retry/sync import
paths). Keeps first price/vat/name, sums quantities on collision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. SQLite order_items overwrite on re-import (VELA CAFE #484669620):
add_order_items, save_orders_batch, mark_order_deleted_in_roa now use
DELETE + INSERT so GoMag quantity changes propagate to dashboard.
2. PL/SQL strict CUI lookup tolerates whitespace (FG COFFE #485065210):
cauta_partener_dupa_cod_fiscal regex ^RO\d → ^RO\s*\d; IN-set uses
canonical v_ro_cui. Platitor/neplatitor business rule preserved.
Python defensive: re.sub whitespace collapse in determine_partner_data.
3. New PJ partners use ANAF official denumire (denumire_override) instead
of GoMag company_name. Existing partners (found by CUI) untouched.
Tests: 18 new (5 SQLite unit, 8 Python unit, 5 Oracle PL/SQL). All green
locally: 228 unit + 26 oracle + 33 e2e.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GoMag vs ROA price comparison generated too many false positives
(kits, volume discounts, special prices). Removes comparison columns,
dots, badges, catalog sync endpoints, and ~950 lines of dead code.
Keeps WRITE path (sync_prices_from_order) for kit pricing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes caused partner_mismatch=1 to loop indefinitely:
1. No-CUI company orders (is_pj=1, no cod_fiscal): old code flagged as
mismatch every cycle. Fixed by requiring new_cf to be non-null for
PF→PJ detection. Stale flags from old code cleared via new
clear_stale_partner_mismatches_no_cui() for out-of-window orders.
2. same_partner resync path did not update cod_fiscal_gomag in SQLite.
On next cycle GoMag returned a CUI but stored_cf was still NULL →
re-detected as mismatch. Fixed by also calling update_partner_resync_data
(not just update_partner_mismatch_batch) in the same_partner branch.
3. GoMag sends CUI with space: 'RO 17922480'. The _strip_ro() regex
^RO left the space → ' 17922480' != '17922480' → false mismatch.
Fixed: changed regex to ^RO\s* and added .strip().
Also adds diagnostic logger.info lines for mismatch detection/resync counts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GoMag sends baseprice (catalog price) alongside price (discounted).
When baseprice > price, the item is volume-discounted — skip ROA
price comparison and show amber "Disc." badge instead of false
mismatch. Strikethrough baseprice in price column for transparency.
Pipeline: parse baseprice → store in SQLite → skip in validation →
pass flag to frontend → render badge (desktop + mobile pill badge
with aria-label, opacity 0.6 for dark mode).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Orders synced before address_mismatch was deployed had stale 0 values,
causing missing blue dots in the dashboard. Adds startup backfill from
stored address JSON + recomputes on each sync for ALREADY_IMPORTED orders.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CUI adjustment (RO prefix change) already implies TVA mismatch, so
suppress redundant TVA badge when CUI badge is shown. Remove
address_mismatch from DIFFS pill count since it's not an ANAF difference.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously the TVA badge/dot appeared whenever ANAF reported neplatitor.
Now it only appears when GoMag CUI prefix (RO=platitor) disagrees with
ANAF status — aligned across JS badges, red dots, SQL filter and count.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove TIER 2 address lookup (county+city without street) from PL/SQL — creates
new address when street differs instead of reusing wrong one
- Replace generic "N diferente" badge with typed micro-badges (CUI, Denumire, TVA,
Adr. livr., Adr. fact., Preturi) with red/amber semantic colors
- Extend addrMatch() regex to strip full Romanian address words (STRADA, NUMAR, BLOC,
COMUNA, SAT, MUNICIPIUL, etc.) — fixes "Strada X" vs "X" false positives
- Extend normalize_company_name() for II, PFA, INTREPRINDERE INDIVIDUALA legal forms
- Persist address_mismatch to SQLite so "Dif." filter includes address-only diffs
- Add red/amber indicator dots to desktop table and mobile list rows
- 12 unit tests for normalization and server-side address matching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Orders imported before the ANAF dedup feature had no anaf_platitor_tva.
Step 4c now auto-backfills on each sync: batch cache lookup, single
ANAF API call for uncached CUIs, bulk DB update in one transaction.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix ANAF API: extract CUI from date_generale (not top-level), fix
notFound casing (capital F)
- Fix missing facturare address when same ID as livrare (copy instead
of skip)
- Replace ANAF cache pre-population stub with real logic (3-month CUIs)
- Restructure order detail modal: inline 2-col GOMAG|ROA layout with
compact address lines replacing collapsed sections
- Fix addrMatch() to use field-level comparison with Romanian
abbreviation stripping (STR, NR, BL, SC, AP, ET, ETAJ, APART)
- Add dashboard "Diferente" filter pill for ANAF-adjusted orders
- Update e2e test for new modal structure
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevent partner duplicates via ANAF CUI verification and dual PL/SQL
search. Fix address matching with street-level comparison and diacritics
normalization. Show partner/address comparison in order detail modal.
- New anaf_service.py: batch ANAF API client with chunking, retry, cache
- PL/SQL: dual CUI search (bare/RO+bare/RO space+bare), 3-tier address
search (street+city+id_loc → city+id_loc → create), strip_diacritics
at storage for addresses and partner names
- SQLite: anaf_cache table, 12 new order columns for partner/address data
- import_service: cod_fiscal_override param, return partner/address from Oracle
- sync_service: ANAF batch integration, denomination mismatch detection,
cache pre-population trigger
- Router: enriched order_detail with partner_info + addresses JSON
- UI: collapsible Detalii Partener + Adrese Comparativ sections in modal,
auto-expand on mismatch, ANAF badges, mobile address cards
- Dashboard: address quality attention indicator
- New scan_duplicate_partners.py script for one-time duplicate audit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add price_match column to SQLite, cached on order detail view
- Background backfill on startup checks all unchecked imported orders
- Extract _enrich_items_with_codmat() helper to deduplicate SKU enrichment
- Attention card now shows same nefacturate count as filter pill
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After saving a SKU mapping, check for SKIPPED orders containing that
SKU and show a floating banner with count + "Importa" button. Batch
retries up to 20 orders and shows result feedback.
Backend:
- get_skipped_orders_with_sku() in sqlite_service.py
- GET /api/orders/by-sku/{sku}/pending endpoint
- POST /api/orders/batch-retry endpoint (max 20, sequential)
Frontend:
- Auto-retry banner after quickMap save with batch import button
- Success/error feedback, auto-dismiss after 15s
Cache-bust: shared.js?v=19
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a "Needs Attention" card above the orders table that surfaces:
- Import errors count (click → ERROR filter)
- Unmapped SKUs count (click → Missing SKUs page)
- Uninvoiced orders >3 days (click → UNINVOICED filter)
Shows green "Totul in ordine" when all metrics are zero.
Backend: add uninvoiced_old count to get_orders() and unresolved_skus
from get_dashboard_stats() to dashboard/orders API response.
Cache-bust: style.css?v=21, dashboard.js?v=29
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Oracle PL/SQL: kit pricing logic with Mode A (distributed discount) and
Mode B (separate discount line), dual policy support, PRETURI_CU_TVA flag
- Eliminate procent_pret from entire stack (Oracle, Python, JS, HTML)
- New settings: kit_pricing_mode, kit_discount_codmat, price_sync_enabled
- Settings UI: cards for Kit Pricing and Price Sync configuration
- Mappings UI: kit badges with lazy-loaded component prices from price list
- Price sync from orders: auto-update ROA prices when web prices differ
- Catalog price sync: new service to sync all GoMag product prices to ROA
- Kit component price validation: pre-check prices before import
- New endpoint GET /api/mappings/{sku}/prices for component price display
- New endpoints POST /api/price-sync/start, GET status, GET history
- DDL script 07_drop_procent_pret.sql (run after deploy confirmation)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add production pricing policy (id_pol_productie) for articles with cont 341/345,
smart discount VAT splitting across multiple rates, per-article id_pol support,
and mapped SKU price validation. Settings UI updated with new controls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sync card was showing previous run data after sync completed because the
last_run query excluded the current run_id even after it finished. Now only
excludes during active running state.
All datetime.now() and SQLite datetime('now') replaced with Europe/Bucharest
timezone to fix times displayed 2 hours behind (was using UTC).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add web_status column to orders table (generic name for platform status)
- Filter cancelled orders during sync, record as CANCELLED in SQLite
- Soft-delete previously-imported cancelled orders in Oracle (if not invoiced)
- Add CANCELLED filter pill + badge in dashboard UI
- New soft_delete_order_in_roa() and mark_order_cancelled() functions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
customer_name was only set on INSERT but not updated on ON CONFLICT,
so re-synced orders kept the old (wrong) customer name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, orders deleted from Oracle (sters=1) remained as IMPORTED
in SQLite, and deleted invoices kept stale cache data. Now the refresh
button and sync cycle re-verify all imported orders against Oracle:
- Deleted orders → marked DELETED_IN_ROA with cleared id_comanda
- Deleted invoices → invoice cache fields cleared
- New status badge for DELETED_IN_ROA in dashboard and logs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- PL/SQL: handle duplicate CODMAT in nom_articole with MAX(id_articol)
- import_service: add explicit conn.rollback() on Oracle errors
- sync_service: auto-fix stale ERROR orders that exist in Oracle
- invoice_service: add data_act (invoice date) from vanzari table
- sync router: new POST /api/dashboard/refresh-invoices endpoint
- order detail: enrich with invoice data (serie, numar, data factura)
- dashboard: refresh invoices button (desktop + mobile icon)
- quick map modal: compact single-row layout, pre-populate existing mappings
- quick map: link on SKU column instead of CODMAT
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parse delivery.total and discounts[] from GoMag JSON into new
delivery_cost/discount_total fields. Add app_settings table for
configuring transport/discount CODMAT codes. When configured,
transport and discount are appended as extra articles in the
Oracle import JSON. Reorder Total column in dashboard/logs tables
and show transport/discount breakdown in order detail modals.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parse order total from GoMag JSON, store in SQLite orders table,
and expose via sync run API. Enables total display in mobile flat rows.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- IMPORTED pill now includes ALREADY_IMPORTED orders in count
- UNINVOICED filter includes ALREADY_IMPORTED orders
- Pill counts (Toate/Importate/Omise/Erori/Nefacturate) always reflect
full period+search, independent of active status filter
- Nefacturate count computed from SQLite cache across full period,
not just current page
- Bootstrap UI: design tokens, soft badge pills, consistent font sizes,
purge inline styles from templates, move badge-pct to style.css
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
orders.status preserves IMPORTED over ALREADY_IMPORTED to avoid
overwriting historical data, so per-run journal views must use
sync_run_orders.status_at_run to show what actually happened in
that specific run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace import_orders (insert-per-run) with orders table (one row per
order, upsert on conflict). Eliminates dedup CTE on every dashboard
query and prevents unbounded row growth at 4-500 orders/sync.
Key changes:
- orders table: PK order_number, upsert via ON CONFLICT DO UPDATE;
COALESCE preserves id_comanda once set; times_skipped auto-increments
- sync_run_orders: lightweight junction (sync_run_id, order_number)
replaces sync_run_id column on orders
- order_items: PK changed to (order_number, sku), INSERT OR IGNORE
- Auto-migration in init_sqlite(): import_orders → orders on first boot,
old table renamed to import_orders_bak
- /api/dashboard/orders: period_days param (3/7/30/0=all, default 7)
- Dashboard: period selector buttons in orders card header
- start.sh: stop existing process on port 5003 before restart;
remove --reload (broken on WSL2 /mnt/e/)
- Add invoice_service, E2E Playwright tests, Oracle package updates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add SSE event bus in sync_service (subscribe/unsubscribe/_emit)
- Add GET /api/sync/stream SSE endpoint for real-time sync progress
- Rewrite logs.html: unified runs table + live feed + summary + filters
- Rewrite logs.js: SSE EventSource client, run selection, pagination
- Dashboard: clickable runs navigate to /logs?run=, sync started banner
- Remove "Import Comenzi" nav item, delete sync_detail.html
- Add error_message column to sync_runs table with migration
- Fix: export TNS_ADMIN as OS env var so oracledb finds tnsnames.ora
- Fix: use get_oracle_connection() instead of direct pool.acquire()
- Fix: CRM_POLITICI_PRET_ART INSERT to match actual table schema
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Flask admin with FastAPI app (api/app/) featuring:
- Dashboard with stat cards, sync control, and history
- Mappings CRUD for ARTICOLE_TERTI with CSV import/export
- Article autocomplete from NOM_ARTICOLE
- SKU pre-validation before import
- Sync orchestration: read JSONs -> validate -> import -> log to SQLite
- APScheduler for periodic sync from UI
- File logging to logs/sync_comenzi_YYYYMMDD_HHMMSS.log
- Oracle pool None guard (503 vs 500 on unavailable)
Test suite:
- test_app_basic.py: 30 tests (imports + routes) without Oracle
- test_integration.py: 9 integration tests with Oracle
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
