diff --git a/.claude/agents/m2d-extractor.md b/.claude/agents/m2d-extractor.md deleted file mode 100644 index 9694cbd..0000000 --- a/.claude/agents/m2d-extractor.md +++ /dev/null @@ -1,216 +0,0 @@ ---- -name: m2d-extractor -description: Extrage date M2D dintr-un screenshot TradeStation. Returnează JSON strict cu schema M2DExtraction (vezi scripts/vision_schema.py). Apelat de /backtest și /batch. -tools: Read, Write -model: opus ---- - -# M2D Vision Extractor - -Ești un extractor specializat pentru screenshot-uri TradeStation M2D. Faci o singură treabă: te uiți la o imagine, scrii un fișier JSON strict + un fișier log, și răspunzi cu un status scurt. Nu chat, nu coaching, nu sugestii. - ---- - -## Inputs - -Caller-ul îți dă: - -1. **`screenshot_path`** — path absolut la PNG/JPG (ex: `D:\PROIECTE\atm-backtesting\screenshots\inbox\2026-05-13-dia-1645.png`). -2. **`screenshot_file`** — basename only (ex: `2026-05-13-dia-1645.png`). Echo-uiești în JSON. -3. *(opțional)* **`hint`** — string scurt de la user (ex: "sell pe US30 5min/1min"). Tratezi ca ipoteză, verifici pe imagine. - -Dacă `screenshot_path` lipsește → scrii o linie în `.log` și te oprești. - ---- - -## Path discipline — STRICT - -Singurele path-uri unde poți scrie: - -- `data/extractions/.json` -- `data/extractions/.log` - -**NU edita**: CSV, `scripts/`, `.claude/`, `screenshots/`, `jurnal.md`, sau orice alt path. Calculezi `basename_no_ext` din `screenshot_file` stripping doar ultima extensie. - -Citești doar screenshot-ul (și opțional `scripts/vision_schema.py` ca referință de schemă dacă te ajută să verifici literal-urile). - ---- - -## Workflow - -### Pas 1 — Citește imaginea - -Folosește `Read` pe `screenshot_path`. Imaginea ajunge ca vizual multimodal. Studiaz-o atent. - -### Pas 2 — Extrage fiecare câmp din `M2DExtraction` - -Schema este în `scripts/vision_schema.py`. Are `extra="forbid"` — orice câmp în plus = rejection. Literal-urile sunt case-sensitive cu diacritice românești. - -| Câmp | Cum citești | -|---|---| -| `screenshot_file` | echo basename primit | -| `data` | timestamp axa X la candle-ul trigger, normalizat `YYYY-MM-DD`. TradeStation folosește MM/DD/YY american; convertești. Nu poate fi în viitor față de UTC azi. | -| `ora_utc` | timpul close al candle-ului trigger convertit din RO local în UTC. Format `HH:MM` (24h). EEST = UTC+3 (vară), EET = UTC+2 (iarnă). Dacă nu ești sigur de sezon → `confidence: low` + pune offset-ul presupus în `ambiguities`. | -| `instrument` | `DIA` dacă preț ~400–500; `US30` dacă preț ~30000–45000; altfel `other`. | -| `directie` | **CRITICAL**: vezi secțiunea "Citirea bulinelor — dot band" mai jos. Pe scurt: `Sell` dacă ultima bulină bright e **light_red** (255,0,0); `Buy` dacă ultima bulină bright e **light_green** (0,255,0). | -| `tf_mare` | exact `5min` sau `15min` — citești din label/overlay TF mare. | -| `tf_mic` | exact `1min` sau `3min` — citești din label chart vizibil. | -| `calitate` | `Clară` (corp candle vizibil, fără wick-uri lungi pe retragere), `Mai mare ca impuls` (corp retragere ≥ corp ultim candle de impuls pe TF mare), `Slabă` (corp mic, wick-uri lungi, indecis), `n/a` dacă retragerea nu e legibilă. | -| `entry` | preț la close-ul candle-ului trigger. Citești de pe axa de preț din dreapta (ground truth peste eventualul label blackbox). | -| `sl` | prețul de pe linia roșie `SL X.XX%`. | -| `tp0`, `tp1`, `tp2` | cele trei niveluri TP desenate de blackbox. TP2 e mereu simetricul SL-ului față de entry. | -| `risc_pct` | procentul de pe label-ul SL (ex: `0.32%` → `0.32`). | -| `outcome_path` | vezi Pas 3. | -| `max_reached` | vezi Pas 3. | -| `be_moved` | vezi Pas 3. | -| `confidence` | `high` dacă tot a fost neambiguu, `medium` dacă ai estimat 1-2 prețuri off-axis, `low` dacă orice câmp required a cerut o presupunere. | -| `ambiguities` | listă scurtă cu ce a fost incert (ex: `["ora_utc DST boundary", "tp1 obscured by overlay"]`). Empty list dacă nimic. | -| `note` | o propoziție scurtă dacă există ceva notabil ce nu se încadrează altundeva. String gol altfel. | - -### Pas 2.5 — Citirea bulinelor — dot band (CRITICAL pentru `directie`) - -Indicatorul blackbox M2D pictează **buline colorate într-o bandă orizontală la baza chart-ului** (TF mic). Aceasta e singura sursă fiabilă pentru direcție și trigger. **NU deduce direcția uitându-te doar la culoarea ultimei candele** — uită-te la dot band. - -**Coordonate dot band** (referință din proiectul ATM, screenshot tipic TradeStation 1919×1032): -- y ≈ 720–760 (banda orizontală de buline) -- citește **de la dreapta spre stânga** — ultima bulină bright = cel mai recent eveniment - -**Paletă fixă** (RGB pure, near-saturation): - -| Bulină | RGB | Rol logic | -|---|---|---| -| **turquoise** (cyan) | (0, 253, 253) | ARM BUY — setup BUY armat pe TF mare | -| **dark_green** | (0, 122, 0) | PRIME BUY — retragere identificată | -| **light_green** | (0, 255, 0) | **TRIGGER BUY** — entry buy aici | -| **yellow** | (253, 253, 0) | ARM SELL — setup SELL armat pe TF mare | -| **dark_red** | (128, 0, 0) | PRIME SELL — retragere identificată | -| **light_red** | (255, 0, 0) | **TRIGGER SELL** — entry sell aici | -| gray | (128, 128, 128) | inactiv / cooldown | - -**Algoritm citire**: - -1. Scanezi dot band-ul **de la dreapta spre stânga** (cele mai recente buline). -2. Identifici **ultima bulină bright** (light_red SAU light_green). Aceasta determină `directie`: - - **light_red** = `Sell` - - **light_green** = `Buy` -3. Verifici **secvența anterioară** (mergi în continuare la stânga): - - Pentru Sell valid: ar trebui să vezi `dark_red` (PRIME) înainte de `light_red` (FIRE), și `yellow` (ARM) chiar mai în spate. - - Pentru Buy valid: `dark_green` înainte de `light_green`, și `turquoise` în spate. -4. Candle-ul **trigger** = candle-ul de pe chart aliniat în timp cu bulina light_red/light_green (poziția X a bulinei pe axa orizontală). - -**Reguli stricte**: -- NU folosi culoarea candelei pentru direcție (candle-urile sunt color-coded și ele, dar bulinele sunt sursa de adevăr). -- Dacă NU vezi clar dot band-ul → `confidence: low` + `ambiguities: ["dot_band_unreadable"]` + best-effort din candle color (cu acest caveat documentat). -- Dacă există un panou OHLC vizibil în screenshot pentru candle-ul trigger, folosește-l ca ground truth pentru `entry` (close). - -### Pas 3 — `outcome_path`, `max_reached`, `be_moved` - -Urmărești ce s-a întâmplat **post-trigger** în screenshot, candle-by-candle. - -**`outcome_path`** (folosește UNICODE arrow `→`, NU `->`) ∈: - -- `SL` — SL atins primul, fără TP înainte -- `TP0→SL` — TP0 atins apoi preț revenit până la SL original (BE NU a fost mutat — loss net) -- `TP0→TP1` — TP0 apoi TP1 atins -- `TP0→TP2` — TP0 apoi TP2 atins -- `TP0→pending` — TP0 atins, trade încă deschis la finalul screenshot-ului -- `pending` — nici SL nici vreun TP atinse până la finalul screenshot-ului - -**`max_reached`** — cel mai înalt nivel **atins de preț**, independent de orice close manual ∈: - -- `SL_first` -- `TP0` -- `TP1` -- `TP2` - -**`be_moved`** — default `true` (rule-enforced per M2D standard: după TP0 muți SL la entry). Set `false` DOAR dacă vezi clar că trade-ul a închis la SL fără BE (i.e. `outcome_path == "TP0→SL"`) sau pentru `outcome_path == "SL"` (TP0 niciodată atins, BE inaplicabil — set `false` în acest caz tot pentru claritate). - -### Pas 4 — Verificare cross-field înainte de write - -Validatorii pydantic vor respinge dacă nu sunt satisfăcute (vezi `scripts/vision_schema.py`): - -1. `entry != sl` -2. Ordering în funcție de `directie`: - - `Buy`: `sl < entry < tp0 < tp1 < tp2` - - `Sell`: `sl > entry > tp0 > tp1 > tp2` -3. `data` nu în viitor (UTC azi). -4. `data` strict `YYYY-MM-DD`; `ora_utc` strict `HH:MM`. -5. `outcome_path == "SL"` ⟹ `max_reached == "SL_first"`. -6. `outcome_path` începe cu `TP0` ⟹ `max_reached ∈ {TP0, TP1, TP2}`. -7. `outcome_path == "pending"` ⟹ orice `max_reached`. - -### Pas 5 — Scrie cele două fișiere - -**`data/extractions/.json`** — JSON pretty-printed, indent 2 spaces, UTF-8, terminator newline. Conține EXACT obiectul M2DExtraction, nimic în plus. - -**`data/extractions/.log`** — format fix: - -``` -[extraction] -image: screenshots/inbox/.png -reasoning: - - identified candle X at coord Y - - read entry from price label "..." - - outcome: TP0 hit at HH:MM, TP1 hit at HH:MM -decisions: - - outcome_path = TP0→TP1 - - max_reached = TP1 -ambiguities: [] -confidence: high -``` - -Adaptezi liniile la ce ai văzut efectiv. `reasoning` are 2-5 bullet points; `decisions` notează deciziile cheie (outcome_path, max_reached, be_moved, confidence). - -Presupui că `data/extractions/` există. Dacă Write eșuează, scrii eroarea în `.log` (dacă poți) și te oprești. - -### Pas 6 — Răspuns final către orchestrator - -După ce ai scris ambele fișiere, returnezi exact un mesaj scurt (max 3 propoziții) în text: - -``` -Extras la ``. Confidence: . Ambiguities: . -``` - -Fără preambul, fără markdown fence cu JSON, fără explicații extra. Caller-ul e un script. - -Dacă screenshot-ul e ILIZIBIL COMPLET, NU abortezi — scrii JSON cu `confidence: low`, `ambiguities: ["image_unreadable"]`, restul câmpurilor best-effort (chiar dacă sunt presupuneri), urmat de `.log` corespunzător, urmat de status-line normal. - ---- - -## Reguli stricte - -1. **NICIODATĂ nu inventezi date**. Dacă un câmp nu e legibil → `confidence: low` + adaugă în `ambiguities`. Estimezi DOAR dacă geometria o permite (TP2 simetric cu SL, TP0 ≈ 0.4·|entry−sl| de la entry, TP1 ≈ 0.6·|entry−sl|). -2. **Axa de preț e ground truth** peste orice label blackbox sau tooltip când diferă. -3. **Nu speculezi despre TF-uri pe care nu le vezi**. Dacă screenshot-ul nu include daily, nu scrii nimic despre trend daily în `note`. -4. **Formatul trebuie să satisfacă `scripts/vision_schema.py` EXACT** — fără câmpuri extra, literal-uri case-sensitive cu diacritice (`Clară`, `Slabă`, `Mai mare ca impuls`). -5. **Un screenshot = un JSON**. Niciodată batch. -6. **Output strict** — fără preambul în răspuns, doar status-line-ul după write. - ---- - -## Exemplu output JSON - -```json -{ - "screenshot_file": "2026-05-13-dia-1645.png", - "data": "2026-05-13", - "ora_utc": "14:45", - "instrument": "DIA", - "directie": "Buy", - "tf_mare": "5min", - "tf_mic": "1min", - "calitate": "Clară", - "entry": 497.42, - "sl": 496.80, - "tp0": 497.67, - "tp1": 497.79, - "tp2": 498.04, - "risc_pct": 0.12, - "outcome_path": "TP0→TP1", - "max_reached": "TP1", - "be_moved": true, - "confidence": "high", - "ambiguities": [], - "note": "" -} -``` diff --git a/.claude/commands/backtest.md b/.claude/commands/backtest.md deleted file mode 100644 index b79fc13..0000000 --- a/.claude/commands/backtest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: Run vision extraction on a single TradeStation screenshot, then append to jurnal CSV + regenerate MD. -argument-hint: " [--calibration]" ---- - -# /backtest — single screenshot vision extraction - -Lansează subagentul `m2d-extractor` pe un screenshot, primește JSON-ul, append la `data/jurnal.csv`, regenerează `data/jurnal.md`. - -## Arguments - -- `$1` (obligatoriu) — path la screenshot. Acceptă: - - basename (`2026-05-13-dia-1645.png`) — caută în `screenshots/inbox/`, fallback `screenshots/processed/` - - path relativ sau absolut explicit -- `--calibration` (flag) — `source=vision_calibration` în loc de `source=vision`. Folosit împreună cu `/m2d-log --calibration` pe același screenshot pentru P4 mismatch report. - -## Workflow - -1. **Rezolvă path-ul** screenshot-ului. Dacă `$1` e doar basename, încearcă `screenshots/inbox/` apoi `screenshots/processed/`. Dacă nu există nicăieri, raportezi eroare și te oprești. - -2. **Invocă subagentul `m2d-extractor`** (definit în `.claude/agents/m2d-extractor.md`) prin Task tool cu `subagent_type: "m2d-extractor"`. Prompt-ul către agent: - - ``` - screenshot_path: - screenshot_file: - ``` - - Agentul scrie `data/extractions/.json` + `.log` și returnează status-line scurt. - -3. **Verifică output-ul**: - - Dacă fișierul `data/extractions/.json` nu există după ce agentul revine → eroare; raportezi și muți screenshot-ul la `screenshots/needs_review/`. - - Citește JSON-ul. Dacă `confidence == "low"` SAU `ambiguities` non-empty cu `image_unreadable` → muți screenshot-ul la `screenshots/needs_review/`, raportezi, nu apelezi append. - -4. **Append la CSV**: - - ```bash - python -c "from pathlib import Path; from scripts.append_row import append_extraction; import json; r = append_extraction(Path('data/extractions/.json'), source=''); print(json.dumps(r, default=str))" - ``` - - `` = `vision_calibration` dacă `--calibration`, altfel `vision`. - - Parsezi răspunsul. Dacă `status == "rejected"`: - - `reason` conține "duplicate" → screenshot deja procesat cu acest source; raportezi și NU îl muți. - - `reason` conține "validation error" → JSON-ul agentului a fost respins; muți screenshot la `screenshots/needs_review/` și raportezi. - - Alte erori → raportezi și lași screenshot-ul unde e. - -5. **Mută screenshot-ul** la `screenshots/processed/` dacă append-ul a reușit și fișierul originar a fost în `inbox/`. Dacă era deja în `processed/`, nu-l muta. - -6. **Regenerează MD**: - - ```bash - python scripts/regenerate_md.py - ``` - -7. **Raport final** (în română): - - ``` - /backtest → trade # adăugat (source=, set=, pl_marius=, confidence=). - Regenerat data/jurnal.md ( rânduri). - ``` - - Dacă screenshot-ul a fost mutat la `needs_review`: - - ``` - /backtest → NEEDS REVIEW: . Mutat la screenshots/needs_review/. - ``` - -## Reguli - -- O singură invocare per screenshot. Nu reapelezi agentul dacă output-ul e dubios — îl muți la `needs_review` și raportezi. -- NU edita CSV direct. -- NU regenera MD dacă append-ul a fost respins. -- Path discipline: subagentul scrie doar la `data/extractions/`; tu (slash command) muți screenshot-uri și apelezi scripts/. diff --git a/.claude/commands/batch.md b/.claude/commands/batch.md deleted file mode 100644 index 4cbf540..0000000 --- a/.claude/commands/batch.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: Procesează toate screenshot-urile din screenshots/inbox/ paralel (5 agenți). Append serial cu partial-failure semantics. -argument-hint: "[N max_parallel=5]" ---- - -# /batch — parallel vision extraction over screenshots/inbox/ - -Procesează screenshot-uri multiple din `screenshots/inbox/`. Lansează până la **5 subagenți `m2d-extractor` în paralel** (cap rigid — context window + rate limits). După ce toți revin, append-ezi rezultatele **serial** (`append_row` citește/scrie CSV — paralelism la write = ID collision garantat). - -## Workflow - -### Fază 1 — Colectează lista - -1. Listează `screenshots/inbox/*.png` (sortat alfabetic). -2. Dacă lista e goală → afișează `Inbox gol. Adaugă PNG-uri în screenshots/inbox/.` și oprește. -3. Dacă `$ARGUMENTS` conține un număr `N`, folosește-l ca `max_parallel` în loc de 5 (dar nu depăși niciodată 5 — hard cap). - -### Fază 2 — Extracție paralelă (max 5 concurent) - -Procesezi în **batch-uri de `max_parallel`**. Pentru fiecare batch: - -- Lansezi câte un Task tool call cu `subagent_type: "m2d-extractor"` pentru fiecare screenshot, ÎN ACELAȘI MESAJ (tool calls paralele). Prompt per agent: - - ``` - Extrage trade din ``. Scrie JSON la `data/extractions/.json` și log la `data/extractions/.log`. - - screenshot_path: - screenshot_file: - ``` - -- Aștepți să se întoarcă toți. Treci la următorul batch. - -**De ce max 5**: peste 5 sub-agenți paraleli începi să saturezi context window-ul orchestratorului cu output-urile lor și rate limits-urile API-ului. Cap rigid. - -### Fază 3 — Append serial cu partial-failure - -Ține trei liste: `ok`, `rejected`, `failed`. Pentru fiecare PNG din lista originală, **în ordine alfabetică**: - -1. **Verifică existența JSON-ului** `data/extractions/.json`: - - Lipsește sau e corupt → mută PNG la `screenshots/needs_review/`, adaugă la `failed` cu motiv `missing/invalid JSON`, continuă. - -2. **Apelează append** (source = `vision` — `/batch` nu suportă calibration, pentru asta folosește `/backtest --calibration` individual): - - ```bash - python -c "from pathlib import Path; from scripts.append_row import append_extraction; import json; r = append_extraction(Path('data/extractions/.json'), source='vision'); print(json.dumps(r, default=str))" - ``` - -3. **Reacționezi**: - - `status == "ok"` → mută PNG la `screenshots/processed/`, adaugă la `ok` cu `id`, `set`, `outcome_path`. - - `status == "rejected"` → mută PNG la `screenshots/needs_review/`, mută JSON la `data/extractions/rejected/.json` (creează folderul dacă lipsește), adaugă la `rejected` cu `reason`. - -4. **NU oprești batch-ul la primul fail**. Continuă până la capăt. - -### Fază 4 — Regenerează MD o singură dată - -```bash -python -m scripts.regenerate_md -``` - -(MD regen după fiecare append e wasteful; CSV-ul e sursa de adevăr.) - -### Fază 5 — Scrie summary la `data/extractions/_batch_.md` - -`` format ISO compact (ex: `2026-05-13T15-45-21Z`). Conținut: - -```markdown -# Batch run - -Total: | OK: | REJECTED: | FAILED: - -## OK -- .png → id=, set=, outcome_path= -- ... - -## REJECTED -- .png — reason: -- ... - -## FAILED -- .png — -- ... -``` - -Secțiuni goale se omit (dacă REJECTED e gol, nu scrii secțiunea). - -### Fază 6 — Afișează summary user-ului - -Format scurt în terminal: - -``` -/batch terminat. Total screenshot-uri. - OK: (trade-uri #, #, ...) - REJECTED: (mutate la screenshots/needs_review/) - FAILED: (mutate la screenshots/needs_review/, JSON lipsă) -Regenerat data/jurnal.md. -Summary scris la data/extractions/_batch_.md. -``` - -## Reguli - -- **Hard cap concurrency la 5**. Chiar dacă `max_parallel` argumentat e mai mare, clamp la 5. -- **Append serial obligatoriu**. `append_extraction` citește CSV, computează `next_id`, scrie atomic; rulat paralel = ID-uri duplicate sau pierderi. -- **Partial failure = continuă**. Un screenshot prost nu blochează restul batch-ului. -- **MD regen o singură dată** la final. -- **Path discipline pentru subagent neschimbată**: agentul scrie doar la `data/extractions/`. Tu (orchestrator) muți screenshot-uri și rejected JSON-uri. -- `/batch` folosește mereu `source=vision`. Pentru calibration, rulează `/backtest --calibration` individual pe fiecare screenshot. diff --git a/.claude/commands/m2d-log.md b/.claude/commands/m2d-log.md deleted file mode 100644 index a1aec7c..0000000 --- a/.claude/commands/m2d-log.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -description: Adaugă rapid un trade în jurnal.csv. 6 câmpuri minim, restul derivate auto. Manual entry rapidă pentru backtest/forward paper. -argument-hint: "[--calibration]" ---- - -# /m2d-log — quick manual M2D entry - -User-ul (Marius) loghează rapid un trade. 6 câmpuri esențiale obligatorii, restul derivate automat de `scripts/manual_log.py`. - -## Workflow - -1. **Parse `$ARGUMENTS`** — flag `--calibration` produce `source=manual_calibration`; altfel `source=manual`. - -2. **Cere user-ului următoarele 6 câmpuri obligatorii** (afișează template-ul de mai jos și roagă să răspundă într-un singur mesaj, format `cheie: valoare`): - - ``` - data: 2026-05-13 (format YYYY-MM-DD) - ora: 17:33 (ora RO local, HH:MM — DST gestionat automat) - dir: Sell (Buy sau Sell) - entry: 492.47 (prețul de intrare = close trigger candle) - sl: 492.77 (SL absolute price) - out: TP0→pending (SL | TP0→SL | TP0→TP1 | TP0→TP2 | TP0→pending | pending) - ``` - - Opționale (utilizatorul le poate omite — defaults aplicate): - ``` - inst: DIA (default DIA — alternative: US30, other) - calitate: Clară (default n/a — Clară | Mai mare ca impuls | Slabă) - tf_mic: 1min (default 1min — alternativ: 3min) - tf_mare: 5min (default 5min — alternativ: 15min) - note: ... (default empty) - ``` - -3. **Parsează răspunsul user-ului**. Tolerant la spații în jurul valorilor. Dacă lipsește un câmp obligatoriu → afișează ce lipsește și ceri din nou. - -4. **Construiește dict-ul** apelând helper-ul Python (printr-un singur `python -c`): - - ```bash - python -c " - import json - from scripts.manual_log import build_extraction - d = build_extraction( - data='', - ora_ro='', - directie='', - entry=, - sl=, - outcome_path='', - instrument='', - tf_mare='', - tf_mic='', - calitate='', - note='', - ) - import pathlib - basename_no_ext = d['screenshot_file'].rsplit('.', 1)[0] - p = pathlib.Path(f'data/extractions/{basename_no_ext}.manual.json') - p.parent.mkdir(parents=True, exist_ok=True) - p.write_text(json.dumps(d, ensure_ascii=False, indent=2), encoding='utf-8') - print(json.dumps({'json_path': str(p), 'screenshot_file': d['screenshot_file']}, default=str)) - " - ``` - - Helper-ul calculează automat: - - `ora_utc` din `ora_ro` (DST-aware Europe/Bucharest) - - `tp0` = entry ± 0.4·|entry−sl| (sign în funcție de directie) - - `tp1` = entry ± 0.6·|entry−sl| - - `tp2` = entry ± |entry−sl| (simetric SL) - - `risc_pct` = 100·|entry−sl|/entry - - `max_reached` din `outcome_path` (SL→SL_first, TP0→SL→TP0, TP0→TP1→TP1, ...) - - `be_moved` din `outcome_path` (False pentru SL/pending, True pentru orice TP0→...) - - `screenshot_file` generat dacă nu e prezent: `--.png` - - Cross-field ordering validat (Buy: sl'), source='') - print(json.dumps(r, default=str)) - " - ``` - -6. **Dacă `status == "ok"`**: - - ```bash - python -m scripts.regenerate_md - ``` - - Apoi afișează concis: - - ``` - ✅ Trade # adăugat — set=, outcome=, pl_marius=, pl_theoretical= - ``` - -7. **Dacă `status == "rejected"`**: - - ``` - ❌ Respins: - ``` - - Dacă `reason` conține "duplicate" → trade-ul cu acel `(screenshot_file, source)` există deja. Dacă vrei să-l suprascrii, șterge linia din `data/jurnal.csv` și re-rulează (sau cere user-ului să specifice `note: ` ca să forțeze basename diferit). - - Dacă `reason` conține "validation" → câmpurile au violat constraint-urile pydantic; reîntrebezi user-ul ce să corecteze. - -8. **Errori în parsing user**: dacă user-ul răspunde ambiguu (ex: lipsește `dir`, sau `entry` nu e număr), afișează ce trebuie corectat și revii la step 2 cu valorile parțiale păstrate. - -## Reguli - -- NU edita CSV direct. -- NU regenera MD pe rejection. -- Helper-ul `build_extraction` ridică `ValueError` dacă: `entry == sl`, `Buy` cu `sl >= entry`, `Sell` cu `sl <= entry`. Propaga eroarea către user cu mesaj clar. - -## Exemple - -**Cel mai scurt input valid**: -``` -data: 2026-05-13 -ora: 17:33 -dir: Sell -entry: 492.47 -sl: 492.77 -out: TP0→pending -``` -→ generează screenshot_file=`2026-05-13-dia-1733.png`, calculează ora_utc=14:33, tp0=492.35, tp1=492.29, tp2=492.17, risc_pct=0.0609, max_reached=TP0, be_moved=True, set=A2 (Mie 17:33). - -**Cu calitate și note**: -``` -data: 2026-05-13 -ora: 17:33 -dir: Sell -entry: 492.47 -sl: 492.77 -out: TP0→TP1 -calitate: Clară -note: bună retragere dimineața, news risc zero -``` - -**Calibrare**: `/m2d-log --calibration` → source=manual_calibration. Folosit cu `/backtest --calibration ` pe același screenshot pentru P4 mismatch report. diff --git a/.claude/commands/stats.md b/.claude/commands/stats.md deleted file mode 100644 index 3572aaa..0000000 --- a/.claude/commands/stats.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: Afișează statistici WR / expectancy / per Set (din data/jurnal.csv). Cu --calibration arată raport P4. -argument-hint: "[--calibration] [--overlay pl_marius|pl_theoretical] [--seed N]" ---- - -# /stats — backtest statistics - -Wrapper read-only peste `scripts/stats.py`. Afișează raportul stat ca atare; eventual adăugă highlight la final dacă observi un Set care îndeplinește pragul STOPPING_RULE. - -## Arguments - -- `--calibration` (flag) — afișează raportul P4 (mismatch field-by-field pe perechi `manual_calibration` ↔ `vision_calibration` join-uite pe `screenshot_file`). -- `--overlay pl_marius|pl_theoretical` (opțional, default `pl_marius`) — care P/L overlay folosește. -- `--seed N` (opțional) — seed pentru bootstrap RNG. Folosește pentru reproducibilitate. - -Default (fără flag-uri): backtest stats — overall + per-Set + per-calitate + per-instrument WR, expectancy, Wilson 95% CI pe WR, bootstrap 95% CI pe expectancy, pe overlay `pl_marius`. - -## Workflow - -1. **Parse `$ARGUMENTS`** și pasează-le direct prin: - - ```bash - python -m scripts.stats $ARGUMENTS - ``` - - `--csv data/jurnal.csv` e default-ul scriptului; nu îl pasezi. - -2. Rulează prin Bash. Output-ul vine pe stdout în UTF-8. - -3. **Afișează output-ul as-is** către user. NU reformata, NU re-rezuma, NU inventa numere. Scriptul are deja format ales (tabele + secțiuni text). - -4. **Highlight STOPPING_RULE** la final, DOAR dacă observi în output un Set care îndeplinește toate cele 3 thresholds din `STOPPING_RULE.md`: - - `N ≥ 40` trade-uri non-pending pe acel Set - - `WR ≥ 55%` cu Wilson 95% CI lower bound ≥ 45% - - `Expectancy ≥ +0.20R` pe overlay `pl_marius` - - Dacă DA pe vreun Set: - - ``` - 🚀 STOPPING RULE: Set îndeplinește pragurile (N=, WR=, Wilson_LB=, E=R). Discută cu user dacă pornește forward paper trading la 0.25R per trade pe acest Set. - ``` - - Dacă **niciun Set** nu îndeplinește toate: nu adăuga highlight. Lasă raportul scriptului să vorbească. - -5. **Highlight calibration P4** (în modul `--calibration`): - - Dacă perechi `(manual_calibration, vision_calibration)` < 10 → adăugă: `Insuficient pentru P4 — continuă să acumulezi calibrare (minim 10 perechi).` - - Dacă ≥ 10 perechi și mismatch rate > 10% pe câmpuri core (`entry/sl/tp0/tp1/tp2/outcome_path/max_reached/directie`) → adăugă: `⚠️ P4 FAIL: mismatch > 10% pe câmpuri core. Fix promptul vision agent (.claude/agents/m2d-extractor.md) și re-rulează calibrarea.` - - Dacă ≥ 10 perechi și mismatch ≤ 10% → adăugă: `✅ P4 PASS: mismatch ≤ 10% pe câmpuri core.` - -6. NU edita CSV. NU regenera MD. Citire pură. - -## Reguli - -- Read-only. Această comandă nu scrie nimic. -- Output-ul scriptului e ground truth — nu inventezi numere; doar le citești și aplici regulile STOPPING_RULE. -- `calitate` e descriptor biased post-outcome (vezi `STOPPING_RULE.md` §3) — raportul îl afișează informational only. NU sugera user-ului să folosească `calitate` ca filtru pentru GO LIVE. -- Highlight-ul `🚀 STOPPING RULE` e doar trigger pentru discuție; decizia GO LIVE rămâne a user-ului, cu caveats-urile semnate în `STOPPING_RULE.md`. diff --git a/.gitignore b/.gitignore index fec7b47..3a5d9ef 100644 --- a/.gitignore +++ b/.gitignore @@ -8,21 +8,9 @@ __pycache__/ venv/ .python-version -# Data — vision extractions and batch logs (regenerable) -data/extractions/*.json -data/extractions/*.log -data/extractions/rejected/* -data/extractions/_batch_*.md -!data/extractions/.gitkeep -!data/extractions/rejected/.gitkeep - -# Screenshots — keep originals out of git (large binary, regenerable jurnal from CSV) -screenshots/inbox/* -screenshots/processed/* -screenshots/needs_review/* -!screenshots/inbox/.gitkeep -!screenshots/processed/.gitkeep -!screenshots/needs_review/.gitkeep +# Excel temp/lock files +~$*.xlsx +*.xlsx.bak # OS / editor .DS_Store diff --git a/0_SYSTEM_PROMPT.md b/0_SYSTEM_PROMPT.md deleted file mode 100644 index deb1a10..0000000 --- a/0_SYSTEM_PROMPT.md +++ /dev/null @@ -1,147 +0,0 @@ -# M2D Backtesting Assistant — System Instructions - -> Acest text se lipește în câmpul **"Custom Instructions"** al proiectului Claude (NU ca fișier knowledge). - ---- - -## Rolul tău -Ești un assistant specializat în backtesting-ul strategiei M2D pură pe US30/DIA. -Utilizatorul (Marius) îți va trimite screenshot-uri din TradeStation cu semnalele unui -indicator "blackbox" custom. Tu reconstruiești trade-uri ipotetice din aceste screenshot-uri -și actualizezi jurnalul de backtest. - -**Nu este vorba de tranzacțiile lui reale** — este backtesting curat al strategiei. -Tu acționezi ca și cum trade-ul s-ar fi executat automat la fiecare semnal valid. - ---- - -## Strategia M2D pe scurt - -### Setup BUY -1. **TF mare** (5min sau 15min): bulină TURQUOISE = semnal direcțional buy -2. **TF mic** (1min sau 3min): bulină VERDE ÎNCHIS = retragere identificată -3. **TF mic**: bulină VERDE DESCHIS = reluare / TRIGGER entry - -### Setup SELL -1. **TF mare**: bulină GALBENĂ = semnal direcțional sell -2. **TF mic**: bulină ROȘU ÎNCHIS = retragere -3. **TF mic**: bulină ROȘU DESCHIS = reluare / TRIGGER entry - -### Reguli SL/TP (calculate automat de blackbox, citite de pe chart) -- **SL** = linia roșie "SL X.XX%" -- **TP0** ≈ 40% din distanța SL (R:R 1:0.4) -- **TP1** ≈ 60% din distanța SL (R:R 1:0.6) -- **TP2** = 100% din distanța SL = SIMETRIC cu SL (R:R 1:1) -- Position size: 1/3 la fiecare zonă -- BE move recomandat: după TP0 atins, muți SL la entry - ---- - -## Ce să faci la fiecare screenshot primit - -### Pas 1: Extrage date din imagine -Identifică și notează exact: -- **Data** (din timestamp axa X, format MM/DD/YY american) -- **Ora trigger entry** (RO, EEST sau EET în funcție de sezon) -- **Instrument** (DIA dacă preț ~497, US30 dacă ~42000, altul dacă diferit) -- **Direcție** (Buy/Sell) -- **TF mare** (deduce din spacing semnal turquoise/galben) -- **TF mic** (chart-ul vizibil — de obicei 1min sau 3min) -- **Preț entry** (preț la trigger candle close) -- **SL preț** + **SL %** (de pe linia roșie) -- **TP0, TP1, TP2 prețuri** + procente -- **Calitate retragere**: - - **Clară** = corp candle vizibil, fără wick-uri lungi - - **Mai mare ca impuls** = corp retragere ≥ corp ultim candle de impuls pe TF mare - - **Slabă** = corp mic, wick-uri lungi, indecis - -### Pas 2: Identifică outcome -Urmărește acțiunea POST-trigger în screenshot: -- Care zone TP au fost atinse? (TP0, TP1, TP2 sau niciuna) -- A fost SL prins înainte? -- Dacă a fost atins TP0 dar apoi reversal — presupun **BE move = Da** (default standard) -- Notează rezultatul: SL / TP0 / TP1 / TP2 - -### Pas 3: Calculează Set (fereastra orară) -- **A1**: 16:35-17:00 RO, Mar/Mie/Joi -- **A2**: 17:00-18:00 RO, Mar/Mie/Joi (sweet spot) -- **A3**: 18:00-19:00 RO, Mar/Mie/Joi -- **B**: 22:00-22:45 RO, Mar/Mie/Joi -- **C - News window**: 15:30-16:30 (orice zi) sau zile FOMC/NFP/CPI -- **D**: Luni sau Vineri -- **Other**: orice altceva - -### Pas 4: Calculează P/L în puncte -Cu logica partial close 1/3 + BE move: -- Hit SL → P/L = -Risc (toată poziția) -- Hit TP0 + BE Da → P/L = +0.133 × Risc -- Hit TP0 + BE Nu → P/L = -0.533 × Risc (LOSS!) -- Hit TP1 + BE Da → P/L = +0.333 × Risc -- Hit TP1 + BE Nu → P/L = 0 (BE) -- Hit TP2 → P/L = +0.667 × Risc (max) - ---- - -## Format output (foarte important) - -Pentru fiecare screenshot, dă răspuns ÎN ACEASTĂ ORDINE și ATÂT (nu adăuga preamble): - -### 1. Rândul de jurnal (markdown table) - -```markdown -| # | Data | Zi | Ora RO | Instrument | Direcție | TF mare | TF mic | Calitate | Entry | SL | TP0 | TP1 | TP2 | Risc % | Hit | BE | P/L Risc | Set | Note | -|---|------|----|----|-----------|----------|---------|--------|----------|-------|-----|------|------|------|--------|-----|----|----|-----|------| -| N | YYYY-MM-DD | [zi] | HH:MM | DIA/US30 | Buy/Sell | 5/15min | 1/3min | Clară/Slabă/Mai mare | XX.XX | XX.XX | XX.XX | XX.XX | XX.XX | 0.XX% | TPx/SL | Da/Nu | +/-X.XX | A1/A2/A3/B/C/D | scurt | -``` - -### 2. Analiză scurtă (2-3 propoziții MAX) - -Format obligatoriu — bifează DA/NU pentru fiecare: -- ✅/❌ **Calitate retragere**: [Clară / Slabă / Mai mare ca impuls] -- ✅/❌ **Fereastră optimă**: [Set X, Tier Y] -- ✅/❌ **News risk**: [există news major ±15 min? Da/Nu] -- **Învățare**: [1 propoziție — ce confirmă/contrazice acest trade?] - -### 3. Cere instrucțiunea de salvat - -La final ÎNTOTDEAUNA întrebi: -> "Adaug rândul la jurnal.md? (răspunde 'da' sau dă-mi instrucțiuni de modificare)" - -DACĂ user-ul răspunde "da", reproduci RÂNDUL FORMATAT MARKDOWN pe care el să-l copy-paste în fișierul `jurnal.md` din proiect. - ---- - -## Reguli stricte - -1. **NICIODATĂ nu inventezi date** — dacă nu vezi clar TP0 în screenshot, scrii "N/A" și ceri user-ului să confirme manual. -2. **NICIODATĂ nu interpretezi semnalele dincolo de ce e vizibil** — nu spui "trend-ul de pe daily era bullish" dacă nu vezi daily-ul. -3. **Format de output identic la fiecare răspuns** — așa user-ul poate copia consistent. -4. **Dacă screenshot-ul e neclar** (calitate slabă, niveluri tăiate), spune ce nu poți citi și cere confirmare. -5. **Nu da sfaturi psihologice / coaching** decât dacă ești întrebat explicit. Tu ești logger + filter checker. -6. **Conversie timezone**: dacă vezi 21:38 în screenshot și nu e explicit RO, presupui că e RO (TradeStation setat pe ora locală). -7. **Numerotare rânduri**: începi de la următorul N după ultimul din jurnal.md (dacă există). Dacă jurnal e gol, începi de la 1. - ---- - -## Workflow tipic - -``` -User: [trimite screenshot] -Tu: [rândul jurnal] + [analiză 3 puncte] + "Adaug la jurnal.md?" -User: "da" -Tu: [reproduci rândul formatat clar pentru copy-paste] -User: [copy-paste în jurnal.md, eventual upload-ează jurnalul actualizat] -``` - -Periodic (la fiecare 10-20 trade-uri), user-ul îți va cere statistici agregate. -Atunci citești tot jurnalul și răspunzi cu: -- WR pe Set -- Hit distribution (SL vs TP0/TP1/TP2) -- WR per calitate retragere -- Net P/L total - ---- - -## Limba - -Răspunde mereu în română. Tu și user-ul vorbiți direct și concis. diff --git a/README.md b/README.md index 8fef9d6..ddb9f26 100644 --- a/README.md +++ b/README.md @@ -1,122 +1,144 @@ -# M2D Backtesting — Setup Proiect Claude +# atm-backtesting -## 📂 Conținut +Jurnal Excel manual pentru backtesting pe semnale blackbox (entry / SL / TP precalculate de alt trader sau de un indicator), cu comparație **5 strategii de management** side-by-side. -Acest ZIP conține tot ce ai nevoie pentru a configura proiectul Claude de backtesting M2D: +## Ce face -| Fișier | Rol | +Introduci datele de bază ale fiecărui trade — dată, oră, strategie, indicator, TF, direcție, SL/TP0/TP1/TP2 în %, outcome — și Excel calculează automat: + +- **R-multiples** și **$ P&L** pentru 5 strategii paralele +- **Dashboard**: Win Ratio, Average Win/Loss, Profit Factor, Risk:Reward, Expectancy, HWM Balance, Max Drawdown — câte un set per strategie +- **Breakdown** per Sesiune, Strategie, Indicator, Direcție +- **Equity curve** (5 linii — câte una per strategie) + +Cele 5 strategii de management comparate: + +| # | Strategie | Comportament | +|---|---|---| +| 1 | **TP0 only** | 100% poziție, close la TP0. Foarte conservator (bird in hand). | +| 2 | **TP1 only** | 100% poziție, OCO la SL sau TP1, fără intervenție. | +| 3 | **TP2 only** | 100% poziție, OCO la SL sau TP2 (let it ride). | +| 4 | **Hybrid + BE** | 50% close la TP0, mut SL la BE, 50% close la TP1. Recomandat de trader. | +| 5 | **Hybrid no BE** | 50% close la TP0, **fără** BE, 50% close la TP1. Compară direct cu #4 ca să vezi dacă BE-ul aduce valoare. | + +Rezultatul: pe aceleași semnale, vezi care metodă de management produce cel mai mare expectancy, profit factor și cea mai mică drawdown. + +### De ce 2 variante Hybrid (cu/fără BE) + +BE move este o **regulă teoretică** de management, nu o decizie istorică pe fiecare trade. Așa că nu o marchezi manual pe rând — în schimb, ambele variante (cu și fără BE) se calculează automat pentru fiecare trade. Compari direct dacă regula BE-ului adaugă R-uri sau le scoate. + +## Setup (o singură dată) + +```powershell +pip install openpyxl +python scripts/generate_template.py +``` + +Se generează `data/backtest.xlsx`. Deschide-l în Excel sau LibreOffice Calc. + +## Workflow zilnic + +1. Deschide `data/backtest.xlsx` și du-te în sheet-ul **Trades**. +2. Adaugă un rând nou (continuă imediat sub ultimul completat — **nu lăsa goluri** între rânduri). +3. Completează coloanele galbene (input) — restul (albastre) se calculează automat: + - **Data**, **Ora RO** (sesiunea + ziua se derivă automat de aici) + - **Strategie** (M2D / EMA cross / ... — dropdown, editabil în Config) + - **Indicator** (DIA / SPY / US30 / ... — dropdown) + - **TF** (1min / 3min / 15min — dropdown; e TF-ul de entry, vezi mai jos) + - **Direcție** (Buy / Sell — dropdown) + - **SL %**, **TP0 %**, **TP1 %**, **TP2 %** — distanțe față de entry, în procente (ex. 0.30 pentru 0.30%) + - **Outcome** (SL / TP0 only / TP1 / TP2 — dropdown) + - **Notes** (opțional) +4. Coloanele albastre derivate (Zi, Sesiune, R_*, $_*, Bal_* pentru cele 5 strategii) se umplu automat. +5. Mergi la sheet-ul **Dashboard** — metricile, breakdown-urile și equity curve se actualizează live. + +### Coloana TF + +Pentru M2D, TF-ul mic este TF-ul de entry, iar TF-ul mare e implicit: + +| TF (input) | Perechea M2D | |---|---| -| `0_SYSTEM_PROMPT.md` | Custom Instructions pentru proiect (lipești textul, NU urci ca knowledge file) | -| `jurnal.md` | Knowledge file principal — jurnalul de backtest (gol, pregătit cu headers) | -| `strategie_M2D.md` | Knowledge file de referință — regulile strategiei | -| `calendar_evenimente.md` | Knowledge file de referință — events economice recurente | -| `README.md` | Acest fișier — instrucțiuni setup | +| 1min | 1 / 5 min | +| 3min | 3 / 15 min | +| 15min | 15 / 60 min | ---- +Dacă folosești o altă strategie cu pereche diferită, descrie în Notes. -## 🚀 Pași de configurare +### Cum se calculează Sesiunea automat -### 1. Creează un proiect nou în Claude.ai -- Mergi la **claude.ai** → click pe **Projects** în sidebar → **+ New Project** -- Nume sugerat: **"M2D Backtest"** -- Descriere (opțional): "Backtesting strategie scalping M2D pe US30/DIA prin analiză screenshot-uri TradeStation blackbox" +Pe baza Data + Ora RO, regulile M2D (vezi `strategie_M2D.md`): -### 2. Setează Custom Instructions -- În proiectul creat, click pe **"Set custom instructions"** -- Deschide `0_SYSTEM_PROMPT.md` din acest ZIP -- Copiază TOT textul (fără secțiunea cu titlul "# M2D Backtesting Assistant — System Instructions" dacă vrei) -- Lipește în câmpul de Custom Instructions -- Save +| Sesiune | Condiție | +|---|---| +| **A1** | 16:35–17:00 RO, Mar/Mie/Joi | +| **A2** | 17:00–18:00 RO, Mar/Mie/Joi (sweet spot) | +| **A3** | 18:00–19:00 RO, Mar/Mie/Joi | +| **B** | 22:00–22:45 RO, Mar/Mie/Joi (Power Hour) | +| **C** | 15:30–16:30 RO (pre-NY chop / news risk), orice zi | +| **D** | Luni sau Vineri | +| **Other** | În afara ferestrelor de mai sus | -### 3. Adaugă fișierele knowledge -- În proiect, click pe **"Add content"** sau **"Project knowledge"** -- Urcă următoarele 3 fișiere: - - `jurnal.md` - - `strategie_M2D.md` - - `calendar_evenimente.md` -- NU urca `0_SYSTEM_PROMPT.md` (acela e doar pentru Custom Instructions) -- NU urca acest `README.md` +> Notă: zilele FOMC/NFP/CPI ar trebui marcate ca C, dar formula nu detectează evenimente — marchează manual în Notes dacă e zi news majoră. -### 4. Test inițial -- Începe o conversație nouă în proiect -- Întreabă: "Ești pregătit să primești screenshot-uri pentru backtest M2D?" -- Răspunsul ar trebui să confirme că înțelege strategia și formatul jurnal +## Configurare ---- +Sheet-ul **Config** permite editarea: -## 📋 Workflow zilnic +- **Account Size Start ($)** — balanța inițială (default $10,000) +- **Risk per Trade (%)** — % din account riscat per trade (default 1.0%) +- Listele pentru dropdown-uri: Strategii, Indicatori, TF, Direcție, Outcome -``` -1. Deschizi un screenshot cu setup M2D din TradeStation blackbox -2. Începi o conversație nouă (sau continui una) în proiectul Claude -3. Atașezi screenshot-ul + scrii "Backtest" -4. Claude îți răspunde cu: - - Rândul de jurnal formatat în markdown - - Analiza scurtă (3 puncte cu ✅/❌) - - Întrebarea "Adaug la jurnal.md?" -5. Răspunzi "da" → Claude îți dă rândul curat de copy-paste -6. Deschizi jurnal.md local (sau de oriunde îl ții) și adaugi rândul -7. La 20+ trade-uri: re-uploadezi jurnal.md actualizat în Project Knowledge +La schimbarea Account Size sau Risk %, toate sumele $ din Trades și Dashboard se recalculează. + +## Formule R-multiples (referință) + +`SL_%`, `TP0_%`, `TP1_%`, `TP2_%` sunt distanțe pozitive față de entry, exprimate în procente. + +Tabelul de mai jos arată R-multiple-ul rezultat pentru fiecare combinație (Outcome × Strategie): + +| Outcome | TP0 only | TP1 only | TP2 only | Hybrid + BE | Hybrid no BE | +|---------|----------|----------|----------|-------------|--------------| +| **SL** | −1 | −1 | −1 | −1 | −1 | +| **TP0 only** | +TP0/SL | −1 | −1 | +0.5·TP0/SL | +0.5·TP0/SL − 0.5 | +| **TP1** | +TP0/SL | +TP1/SL | −1 | +0.5·(TP0+TP1)/SL | +0.5·(TP0+TP1)/SL | +| **TP2** | +TP0/SL | +TP1/SL | +TP2/SL | +0.5·(TP0+TP1)/SL | +0.5·(TP0+TP1)/SL | + +**Citirea Outcome-ului**: +- `SL` — prețul a atins SL fără să atingă vreodată TP0 (loss complet). +- `TP0 only` — prețul a atins TP0, dar nu și TP1 (ulterior fie a venit înapoi la SL, fie a fost închis la BE pentru variantele cu BE move). +- `TP1` — prețul a atins TP1 (a trecut prin TP0). +- `TP2` — prețul a atins TP2 (a trecut prin TP0 și TP1). + +**Asumpții de simulare**: +- `TP1 only` și `TP2 only` simulează OCO pur, fără intervenție manuală. Outcome=`TP0 only` se închide la SL (presupunere worst-case). +- `TP2 only` cu Outcome=`TP1` se închide la SL (TP1 a fost atins, dar SL ar fi venit înainte de TP2). +- Diferența dintre Hybrid + BE și Hybrid no BE apare doar când Outcome=`TP0 only`; la TP1/TP2 ambele dau identic. + +## Regenerare template + +Dacă strici structura Excel-ului accidental sau modifici `scripts/generate_template.py`: + +```powershell +python scripts/generate_template.py ``` ---- +Atenție: **suprascrie** `data/backtest.xlsx`. Fă backup la rândurile tale înainte (copy-paste într-un alt fișier sau export CSV). -## 🔄 Sincronizare jurnal +## Decizii GO LIVE / ABANDON -Claude vede doar versiunea de `jurnal.md` care e în Project Knowledge. Deci: +Vezi `STOPPING_RULE.md` — threshold-urile semnate (N≥40, WR≥55%, Expectancy≥+0.20R) și caveat-urile metodologice. -- **După fiecare câteva trade-uri** (ex: 5-10): re-uploadează `jurnal.md` actualizat -- **Înainte să ceri statistici**: ÎNTOTDEAUNA re-uploadează jurnalul, altfel Claude lucrează cu date vechi - -**Sfat practic**: ține `jurnal.md` într-un loc ușor accesibil (Google Drive, Dropbox, Gitea-ul tău, sau local cu sync). Mai ușor de updatat. - ---- - -## 📊 Cereri tipice pe care le poți face - -După ce ai 20+ trade-uri în jurnal: +## Fișiere ``` -"Dă-mi statisticile complete: WR pe Set, hit distribution, WR per calitate retragere" +atm-backtesting/ +├── data/ +│ └── backtest.xlsx # source of truth — jurnalul tău +├── scripts/ +│ └── generate_template.py # regenerator template +├── strategie_M2D.md # referință reguli M2D (Buy/Sell setup, SL/TP, sesiuni) +├── calendar_evenimente.yaml # calendar news (FOMC/NFP/CPI etc.) pentru identificare manuală sesiune +├── STOPPING_RULE.md # threshold-uri decizie + caveats semnate +├── pyproject.toml # dependență: openpyxl +└── README.md # acest fișier ``` - -``` -"Care e Set-ul cu cel mai mare expectancy? Și cel mai prost?" -``` - -``` -"Pe ultimele 30 trade-uri, ce procent au fost cu calitate retragere Slabă? -Au influențat negativ WR-ul?" -``` - -``` -"Compară performanța pe DIA vs US30" -``` - -``` -"Sunt câteva trade-uri Set C (news days). Dacă le exclud, cum se schimbă WR-ul global?" -``` - ---- - -## ⚠️ Limitări de știut - -1. **Claude nu poate edita direct fișiere din Project Knowledge** — tu actualizezi local și re-uploadezi -2. **Calitatea outputului depinde de calitatea screenshot-ului** — dacă liniile TP/SL sunt tăiate, Claude va cere clarificări -3. **Recunoașterea bulinelor colorate** poate avea greșeli ocazionale — verifică primele 5-10 trade-uri să fii sigur că Claude interpretează corect -4. **Pentru trade-uri ambigue** (semnal neclar, multiple posibilități), Claude îți cere confirmare în loc să presupună - ---- - -## 🎯 Obiectiv backtest - -- **Pragul minim de date**: 50 trade-uri pentru concluzii inițiale -- **Pragul de încredere**: 200+ trade-uri pentru statistici robuste -- **Întrebări la care vrei să răspunzi**: - - Care Set are cel mai mare expectancy? - - WR > 55% pe Set A2 (sweet spot)? - - Filtrul de calitate retragere are impact real? - - News days chiar trebuie excluse? - - DIA sau US30 e mai bun pentru M2D? - -Mult succes la backtest! 🚀 diff --git a/STOPPING_RULE.md b/STOPPING_RULE.md index ab72bfa..a7e6088 100644 --- a/STOPPING_RULE.md +++ b/STOPPING_RULE.md @@ -1,6 +1,6 @@ -# STOPPING_RULE — M2D Backtesting +# STOPPING_RULE — Backtesting jurnal Excel -**Versiune**: 1 +**Versiune**: 2 **Data**: 2026-05-13 **Status**: SIGNED — Marius @@ -8,34 +8,35 @@ ## Întrebarea de decis -Pentru fiecare Set candidat (A1, A2, A3, B), decidem una din trei: +Pentru fiecare combinație (Indicator × Sesiune) candidată, decidem una din trei: - **GO LIVE** — pornesc forward paper trading cu 0.25R per trade (validare reală) -- **EXTEND COLLECTION** — mai colectez screenshot-uri, încă nu sunt date suficiente -- **ABANDON** — strategia nu are edge măsurabil pe acest Set; renunț la Set sau la întreaga strategie +- **EXTEND COLLECTION** — mai colectez trade-uri, încă nu sunt date suficiente +- **ABANDON** — strategia nu are edge măsurabil pe această combinație + +Decizia se ia separat pentru fiecare strategie de management (Hybrid 50/50+BE / TP1-only OCO / TP2-only OCO) — Dashboard-ul din `data/backtest.xlsx` afișează metrici side-by-side. --- -## Threshold-uri obligatorii pentru GO LIVE pe un Set +## Threshold-uri obligatorii pentru GO LIVE pe o combinație Indicator × Sesiune -Toate condițiile trebuie satisfăcute simultan: +Toate condițiile trebuie satisfăcute simultan, pentru cel puțin o strategie de management: -1. **N ≥ 40** trade-uri non-pending pe acel Set -2. **WR ≥ 55%** (Wilson 95% CI lower bound ≥ 45%) -3. **Expectancy ≥ +0.20R** pe overlay-ul `pl_marius` (50% TP0 + BE + close ~TP1) -4. **Calibration P4 PASS** — pe primele 10 trade-uri double-extracted (manual + vision), mismatch rate ≤10% pe câmpuri core (`entry`, `sl`, `tp0/1/2`, `outcome_path`, `max_reached`, `directie`) +1. **N ≥ 40** trade-uri pe acea combinație (vezi Dashboard → PER SESIUNE / PER INDICATOR) +2. **WR ≥ 55%** (rule-of-thumb: 95% CI lower bound trebuie să fie ≥ 45% — calculează manual sau folosește calculator Wilson extern) +3. **Expectancy ≥ +0.20R** pe strategia aleasă (Hybrid 50/50+BE implicit; alternativ TP1-only OCO sau TP2-only OCO) -Dacă oricare condiție pică → **EXTEND COLLECTION** sau **ABANDON** (vezi mai jos). +Dacă oricare condiție pică → **EXTEND COLLECTION** sau **ABANDON**. --- -## Threshold-uri pentru ABANDON pe un Set +## Threshold-uri pentru ABANDON Oricare e suficient: -- N ≥ 40 și WR < 45% → edge negativ; ABANDON acest Set -- N ≥ 40 și Expectancy ≤ −0.10R → ABANDON -- Wilson 95% CI lower bound stabil sub 50% după N ≥ 60 → ABANDON +- N ≥ 40 și WR < 45% pe toate cele 3 strategii → edge negativ; ABANDON combinația +- N ≥ 40 și Expectancy ≤ −0.10R pe toate cele 3 strategii → ABANDON +- WR observat stabil sub 50% după N ≥ 60 → ABANDON --- @@ -45,23 +46,44 @@ Oricare e suficient: 1. **N=40 = directional evidence, NU scientific proof**. Intervalul de încredere 95% pentru WR la N=40, WR observat 55%, este aproximativ [40%, 70%]. "Validated" la N=40 înseamnă "merită să tradez cu 0.25R", NU "edge confirmat statistic". Confirmarea reală vine din forward paper trading. -2. **Selection bias rezidual**. Chiar și cu scroll protocol (vezi `WORKFLOW.md`), eu am ales perioada pe care scroll-uiesc. Acest bias e parțial mitigat, NU eliminat. +2. **Selection bias rezidual**. Chiar dacă păstrez disciplina să loghez toate trigger-urile (inclusiv loss-urile clare), eu aleg ce perioadă scroll-uiesc. Acest bias e parțial mitigat prin trade-by-trade logging, NU eliminat. -3. **Lookahead bias pe `calitate`**. Câmpul `calitate ∈ {Clară, Mai mare ca impuls, Slabă}` este clasificat post-outcome (am văzut chart-ul întreg). DECI: NU folosesc `calitate` ca filtru în stopping rule. Rămâne descriptor în jurnal, NU criteriu de tradare. +3. **Lookahead bias pe calitate subiectivă**. Nu mai există coloană `calitate` în jurnalul Excel — clasificarea subiectivă post-outcome a fost eliminată ca sursă de bias. Note-urile rămân pentru context, dar NU sunt folosite ca filtru de decizie. -4. **Backtest = upper bound al expectancy real**. Execuția live va avea slippage, latențe, emoții. Expectancy real probabil 0.05-0.15R sub backtest. De aceea pragul `+0.20R` în backtest = aproximativ break-even-cu-edge-mic în live. +4. **Backtest = upper bound al expectancy real**. Execuția live va avea slippage, latențe, emoții, mouse-trip-uri. Expectancy real probabil 0.05–0.15R sub backtest. De aceea pragul `+0.20R` în backtest ≈ break-even-cu-edge-mic în live. -5. **Indicator drift**. Dacă indicatorul blackbox se update-ează, trade-urile vechi devin istoric irelevant. Trackuit prin coloana `indicator_version` în CSV; reset stat-uri la schimbare. +5. **Indicator drift**. Dacă indicatorul blackbox se update-ează, trade-urile vechi devin istoric irelevant. Trackează manual versiunea indicatorului în coloana Notes; reset stats la schimbare semnificativă. + +--- + +## Comparația de strategii — context + +Cele 5 overlay-uri calculate automat per trade în `Trades` sheet: + +| Strategie | Descriere | +|---|---| +| **TP0 only** | 100% poziție, close la TP0. Maxim conservator. | +| **TP1 only** | 100% poziție, OCO la SL sau TP1, fără intervenție manuală. | +| **TP2 only** | 100% poziție, OCO la SL sau TP2 (let it ride). | +| **Hybrid + BE** | 50% close la TP0, mut SL la BE, 50% close la TP1. Recomandat de trader. | +| **Hybrid no BE** | 50% close la TP0, fără BE, 50% close la TP1. Control pentru izolarea valorii BE-ului. | + +**Decizie de management se ia DUPĂ ce ai N≥40 trade-uri**: alegi strategia cu cel mai bun Expectancy + Profit Factor pe combinația de Indicator × Sesiune care a trecut gate-urile. + +**Asumpții de simulare**: +- `TP1 only` și `TP2 only` simulează OCO pur, fără intervenție; Outcome=`TP0 only` se închide la SL. +- `TP2 only` cu Outcome=`TP1`: TP2 nu a fost atins → presupunem că, în lipsa time-stop-ului, SL ar fi venit ulterior; tratat ca −1R. Asumpția subestimează ușor TP2-only dacă în realitate trade-ul s-ar fi închis profitabil prin alt mecanism, dar e prudent. +- BE este parametru teoretic — nu un input per trade. Hybrid + BE vs Hybrid no BE diferă strict când Outcome=`TP0 only` (diferența = +0.5R în favoarea BE). --- ## Post-GO LIVE protocol -După un Set primește GO LIVE: +După o combinație Indicator × Sesiune × Strategie primește GO LIVE: -1. Forward paper trading cu 0.25R per trade pe acel Set. +1. Forward paper trading cu 0.25R per trade. 2. Minimum 20 trade-uri live înainte de a urca sizing-ul la 0.5R sau full. -3. Dacă WR live diverge >10pp de backtest în prima 20 trade-uri → review (probabil execuție defectă sau bias subestimat). +3. Dacă WR live diverge >10pp de backtest în primele 20 trade-uri → review (probabil execuție defectă sau bias subestimat). --- @@ -71,4 +93,4 @@ După un Set primește GO LIVE: Marius — semnat prin commit git — data: 2026-05-13 ``` -Toate cele 5 caveats înțelese și acceptate. Procedez cu calibrarea P4 pe `dia-1min-example.png`. +Toate cele 5 caveats înțelese și acceptate. diff --git a/WORKFLOW.md b/WORKFLOW.md deleted file mode 100644 index 6798567..0000000 --- a/WORKFLOW.md +++ /dev/null @@ -1,86 +0,0 @@ -# WORKFLOW — colectare screenshot-uri M2D - -**Scop**: minimizează **selection bias** când colectezi screenshot-uri istorice din TradeStation. - -> **Premisa ascunsă pe care o atacăm**: dacă faci screenshot doar la trade-urile care "arată interesant", WR-ul măsurat va fi structural mai mare decât WR-ul real. Setup-urile care arată "dubios" la trigger sunt mai des losers — dacă le filtrezi inconștient, statisticile mint. - ---- - -## Regula de aur - -> **Dacă vezi o bulină verde-deschis (BUY trigger) sau roșu-deschis (SELL trigger) pe TF mic care urmează unei bulin verde-închis/roșu-închis după turquoise/galben pe TF mare → screenshot, indiferent cum arată setupul.** - -NU evaluezi calitatea ÎNAINTE de screenshot. Calitatea o pune extractorul (manual sau vision) pe baza imaginii, nu tu pe baza memoriei. - ---- - -## Scroll protocol - -### 1. Alege perioada în avans - -- Definește o fereastră calendaristică concretă: "Mar/Mie/Joi între 16:35-18:00 RO, din 2025-09-01 până în 2025-12-31". -- **Scrie perioada aleasă într-un comentariu sau fișier înainte de a începe scroll-ul**. Așa nu poți extinde sau restrânge perioada în funcție de ce vezi. - -### 2. Configurare TradeStation - -- TF mic vizibil (1min sau 3min). -- TF mare overlay sau pe ecran separat — important să vezi semnalul direcțional. -- Indicator blackbox activ pe ambele TF-uri. -- Zoom suficient ca să distingi culorile bulinelor clar. - -### 3. Scroll candle-by-candle - -- Începi cu prima zi a perioadei, ora 16:35 (sau începutul fereastrei alese). -- Avansezi candle-by-candle (sau cu pas mic). **NU sări** peste perioade lungi "pentru că arată plat" — chiar și acolo pot fi trigger-e ratate. -- La fiecare bulină verde-deschis / roșu-deschis pe TF mic în fereastra orară a perioadei → **STOP. Screenshot.** -- Verifică post-screenshot că setup-ul respectă pre-condițiile (turquoise/galben pe TF mare anterior, retragere intermediară). Dacă NU respectă → este un trigger fără setup valid; documentează cu nume `-invalid.png` (sau șterge), dar **logează decizia în WORKFLOW_LOG.md de mai jos**. - -### 4. Workflow log (audit trail anti-bias) - -Creează un fișier `data/workflow_log.md` la pornirea fiecărei sesiuni de scroll: - -```markdown -## 2026-05-13 — sesiune scroll perioada 2025-09-01 → 2025-09-15 - -- Start: 14:32 RO -- Perioada scroll-uită: 2025-09-01 → 2025-09-15, Mar/Mie/Joi, 16:35-18:00 -- Trigger-e găsite: 12 -- Screenshot-uri făcute: 12 -- Screenshot-uri NEFĂCUTE (cu motiv): - - 2025-09-04 17:14 — bulina verde-deschis dar fără verde-închis înainte → NU e M2D valid (trigger fals) -- End: 15:48 RO -``` - -**Regula**: dacă rata screenshot/trigger < 95% pe o sesiune, ai un motiv documentat pentru cele lipsă. Dacă nu — ai un bias. - -### 5. Cazuri ambigue (cum eviți cherry-picking) - -- "Nu sunt sigur că e M2D" → screenshot oricum, log la `data/workflow_log.md` ca "ambiguous: motiv". Extractorul (manual/vision) decide final. -- "Imaginea e neclară" → screenshot oricum; vision poate să returneze `confidence:low` și merge la `needs_review/`. -- "Setupul arată slab" → IRRELEVANT la momentul screenshot. Screenshot. Calitate o pune extractorul. (Da, calitate e descriptor biased — vezi STOPPING_RULE.md punct 3 — dar NU folosim calitate ca filtru.) -- "Trade-ul a pierdut clar" → IRRELEVANT. Screenshot. Asta e exact biasul pe care îl evităm. - ---- - -## Anti-pattern-uri (NU FACE asta) - -- ❌ "Scrol până găsesc un trade frumos" — biased. -- ❌ "Sar peste ziua asta, n-a fost nimic interesant" — biased. -- ❌ "Refac screenshot-ul, primul a ieșit prost" → ok dacă primul e ilizibil; NU ok dacă vrei un screenshot "mai clar" pe un trade winning. -- ❌ "Văd că e SL clar, nu merită screenshot" → exact opusul a ce vrei. - ---- - -## Calibration trades (primele 10) - -Pentru cele 10 trade-uri de calibrare (P4 gate): - -- **Tu** (Marius) extragi manual TOATE câmpurile prin `/m2d-log` (source=`manual_calibration`). -- **Apoi** rulezi `/backtest screenshot.png` pentru extracție vision (source=`vision_calibration`). -- `/stats --calibration` compară field-by-field. -- Acceptance: ≤10% mismatch pe câmpurile core. >10% → fix promptul vision agent (`.claude/agents/m2d-extractor.md`) și re-rulează. - ---- - -## Versiune -- v1 (2026-05-13) — draft inițial diff --git a/calendar_evenimente.md b/calendar_evenimente.md deleted file mode 100644 index d74cb67..0000000 --- a/calendar_evenimente.md +++ /dev/null @@ -1,38 +0,0 @@ -# Calendar evenimente economice — recurente - -## Lunar - -| Eveniment | Frecvență | Ora RO | Impact | -|---|---|---|---| -| **NFP** | Prima vineri din lună | 15:30 | EXTREM | -| **CPI** | Mid-luna (10-15) | 15:30 | EXTREM | -| **PPI** | După CPI | 15:30 | MARE | -| **Retail Sales** | ~15 lună | 15:30 | MARE | -| **PCE Price Index** | Ultimele zile lună | 15:30 | MARE | -| **ADP Employment** | Miercuri pre-NFP | 15:15 | MEDIU | -| **JOLTS Job Openings** | 1-a săptămână | 17:00 | MEDIU | -| **ISM Manuf/Services** | 1-a săptămână | 17:00 | MEDIU | - -## Săptămânal - -| Eveniment | Zi | Ora RO | Impact | -|---|---|---|---| -| **EIA Crude Oil Inventories** | Miercuri | 17:30 | MEDIU pe US30 (indirect) | -| **Initial Jobless Claims** | Joi | 15:30 | MIC-MEDIU | - -## FOMC 2026 (8 ședințe/an, miercuri 21:00 RO, *cu excepții DST*) - -| Data | Comentariu | -|---|---| -| 28 ianuarie | Statement 21:00 RO + Powell 21:30 | -| 18 martie | ATENȚIE: 20:00 RO (Europa pe EET, US deja DST) + SEP/dot plot | -| 29 aprilie | 21:00 RO + Powell 21:30 | -| 17 iunie | + SEP/dot plot | -| 29 iulie | 21:00 RO | -| 16 septembrie | + SEP/dot plot | -| 28 octombrie | 21:00 RO | -| 9 decembrie | + SEP/dot plot | - -## Regulă pentru backtesting - -**Trade-urile din ferestrele ± 15 min față de NFP/CPI/FOMC** → marcate ca **Set C - News window** și analizate SEPARAT, NU în statistica principală. diff --git a/data/_meta.yaml b/data/_meta.yaml deleted file mode 100644 index 1404385..0000000 --- a/data/_meta.yaml +++ /dev/null @@ -1,18 +0,0 @@ -schema_version: 1 - -# Versions stamped on every CSV row. Update only when behavior changes. -indicator_version: "v-2026-05" -pl_overlay_version: "marius-v1" # 50% TP0 + BE + close ~TP1 - -csv_schema_version: 1 -calendar_schema_version: 1 - -# Tier 1 / Tier 2 trading windows (informational; Set calc uses calendar_parse.py logic) -tier_1_window: - start_ro: "16:35" - end_ro: "18:00" - days: ["Tue", "Wed", "Thu"] -tier_2_window: - start_ro: "22:00" - end_ro: "22:45" - days: ["Tue", "Wed", "Thu"] diff --git a/data/backtest.xlsx b/data/backtest.xlsx new file mode 100644 index 0000000..5240a82 Binary files /dev/null and b/data/backtest.xlsx differ diff --git a/data/extractions/.gitkeep b/data/extractions/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/data/extractions/rejected/.gitkeep b/data/extractions/rejected/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/data/jurnal.md b/data/jurnal.md deleted file mode 100644 index 1165f61..0000000 --- a/data/jurnal.md +++ /dev/null @@ -1,3 +0,0 @@ -# Jurnal M2D (auto-generated) - -*Niciun trade încă. Adaugă unul prin `/m2d-log` sau `/backtest`.* diff --git a/dia-1min-example.png b/dia-1min-example.png deleted file mode 100644 index c6dc99b..0000000 Binary files a/dia-1min-example.png and /dev/null differ diff --git a/dia-1min-example2.png b/dia-1min-example2.png deleted file mode 100644 index f7fa6ad..0000000 Binary files a/dia-1min-example2.png and /dev/null differ diff --git a/jurnal.md b/jurnal.md deleted file mode 100644 index bcc0b2f..0000000 --- a/jurnal.md +++ /dev/null @@ -1,40 +0,0 @@ -# Jurnal Backtest M2D — US30 / DIA - -**Strategie**: M2D pură (2 timeframe-uri, semnale blackbox TradeStation) -**Începută**: [completează data primului trade] -**Status**: backtest în derulare - ---- - -## Statistici curente - -> Acest sumar se actualizează manual la fiecare ~10 trade-uri. - -- **Total trade-uri**: 0 -- **Win Rate**: - -- **Net P/L (% Risc)**: - -- **Ultimul update**: - - ---- - -## Trade-uri - -| # | Data | Zi | Ora RO | Instrument | Direcție | TF mare | TF mic | Calitate | Entry | SL | TP0 | TP1 | TP2 | Risc % | Hit | BE | P/L Risc | Set | Note | -|---|------|----|----|-----------|----------|---------|--------|----------|-------|-----|------|------|------|--------|-----|----|----|-----|------| -| | | | | | | | | | | | | | | | | | | | | - ---- - -## Note generale (observații din backtest) - -- - ---- - -## Excluderi din statistici - -Trade-uri care au fost identificate dar excluse din analiza principală (de obicei zile news, condiții anormale): - -| # | Motiv excludere | -|---|----------------| -| | | diff --git a/pyproject.toml b/pyproject.toml index 2718738..298fa57 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,25 +1,11 @@ [project] name = "atm-backtesting" -version = "0.1.0" -description = "M2D backtesting system — vision extraction + stats" +version = "0.2.0" +description = "Jurnal Excel manual pentru backtesting M2D + comparație 3 strategii de management (Hybrid 50/50+BE / TP1-only OCO / TP2-only OCO)" requires-python = ">=3.11" dependencies = [ - "pydantic>=2.5", - "pyyaml>=6.0", - "scipy>=1.11", - "numpy>=1.26", + "openpyxl>=3.1", ] -[project.optional-dependencies] -dev = [ - "pytest>=7.4", - "pytest-cov>=4.1", -] - -[tool.pytest.ini_options] -testpaths = ["tests"] -python_files = ["test_*.py"] -addopts = "-ra -q" - [tool.setuptools] packages = ["scripts"] diff --git a/screenshots/inbox/.gitkeep b/screenshots/inbox/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/screenshots/needs_review/.gitkeep b/screenshots/needs_review/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/screenshots/processed/.gitkeep b/screenshots/processed/.gitkeep deleted file mode 100644 index e69de29..0000000 diff --git a/scripts/append_row.py b/scripts/append_row.py deleted file mode 100644 index 6e1202a..0000000 --- a/scripts/append_row.py +++ /dev/null @@ -1,311 +0,0 @@ -"""Append a validated M2D extraction to ``data/jurnal.csv``. - -Pipeline: - JSON file --> pydantic validate (M2DExtraction) - --> load data/_meta.yaml (versions) - --> compute id, ora_ro, zi, set, pl_marius, pl_theoretical, extracted_at - --> dedup on (screenshot_file, source) - --> atomic CSV write (sibling .tmp + os.replace) - -Source values - - ``vision`` : produced by the vision subagent - - ``manual`` : Marius logged by hand - - ``manual_calibration`` : calibration P4 — manual leg - - ``vision_calibration`` : calibration P4 — vision leg - -A row with ``source=manual_calibration`` and a row with ``source=vision_calibration`` -for the *same* screenshot are allowed to coexist (different dedup keys). - -Failure mode: ``append_extraction`` NEVER raises. On any error (missing JSON, -pydantic ValidationError, dedup hit, etc.) it returns -``{"status": "rejected", "reason": "...", "id": None, "row": None}`` so the -caller (a slash command) can decide what to do with the screenshot -(move to ``needs_review/``, log to workflow, etc.). -""" - -from __future__ import annotations - -import csv -import json -import os -import traceback -from datetime import datetime, timezone -from pathlib import Path -from typing import Any, Literal - -import yaml -from pydantic import ValidationError - -from scripts.calendar_parse import calc_set, load_calendar, utc_to_ro -from scripts.pl_calc import pl_marius, pl_theoretical -from scripts.vision_schema import M2DExtraction, parse_extraction - -__all__ = [ - "CSV_COLUMNS", - "VALID_SOURCES", - "ZI_RO_MAP", - "csv_columns", - "append_extraction", -] - - -Source = Literal["vision", "manual", "manual_calibration", "vision_calibration"] - -VALID_SOURCES: frozenset[str] = frozenset( - {"vision", "manual", "manual_calibration", "vision_calibration"} -) - - -# Canonical column order (29) — must stay stable; regenerate_md + stats depend on it. -CSV_COLUMNS: tuple[str, ...] = ( - "id", - "screenshot_file", - "source", - "data", - "zi", - "ora_ro", - "ora_utc", - "instrument", - "directie", - "tf_mare", - "tf_mic", - "calitate", - "entry", - "sl", - "tp0", - "tp1", - "tp2", - "risc_pct", - "outcome_path", - "max_reached", - "be_moved", - "pl_marius", - "pl_theoretical", - "set", - "indicator_version", - "pl_overlay_version", - "csv_schema_version", - "extracted_at", - "note", -) - - -ZI_RO_MAP: dict[str, str] = { - "Mon": "Lu", - "Tue": "Ma", - "Wed": "Mi", - "Thu": "Jo", - "Fri": "Vi", - "Sat": "Sa", - "Sun": "Du", -} - - -def csv_columns() -> list[str]: - """Return the 29-column header in canonical order.""" - return list(CSV_COLUMNS) - - -# --------------------------------------------------------------------------- -# helpers -# --------------------------------------------------------------------------- - - -def _load_meta(meta_path: Path) -> dict[str, Any]: - with meta_path.open("r", encoding="utf-8") as fh: - meta = yaml.safe_load(fh) or {} - required = ("indicator_version", "pl_overlay_version", "csv_schema_version") - missing = [k for k in required if k not in meta] - if missing: - raise ValueError(f"_meta.yaml missing required keys: {missing}") - return meta - - -def _read_existing_rows(csv_path: Path) -> list[dict[str, str]]: - if not csv_path.exists() or csv_path.stat().st_size == 0: - return [] - with csv_path.open("r", encoding="utf-8", newline="") as fh: - reader = csv.DictReader(fh) - return list(reader) - - -def _next_id(rows: list[dict[str, str]]) -> int: - max_id = 0 - for r in rows: - raw = r.get("id", "") - if not raw: - continue - try: - v = int(raw) - except (TypeError, ValueError): - continue - if v > max_id: - max_id = v - return max_id + 1 - - -def _format_optional(value: float | None) -> str: - return "" if value is None else f"{value:.4f}" - - -def _write_csv_atomic( - csv_path: Path, rows: list[dict[str, str]], columns: list[str] -) -> None: - csv_path.parent.mkdir(parents=True, exist_ok=True) - tmp = csv_path.with_suffix(csv_path.suffix + ".tmp") - with tmp.open("w", encoding="utf-8", newline="") as fh: - writer = csv.DictWriter(fh, fieldnames=columns) - writer.writeheader() - for row in rows: - writer.writerow({k: row.get(k, "") for k in columns}) - os.replace(tmp, csv_path) - - -def _build_row( - extraction: M2DExtraction, - *, - source: str, - row_id: int, - meta: dict[str, Any], - calendar: list[dict[str, Any]], - extracted_at: str, -) -> dict[str, str]: - d_ro, t_ro, day_short = utc_to_ro(extraction.data, extraction.ora_utc) - set_label = calc_set(d_ro, t_ro, day_short, calendar) - pl_m = pl_marius(extraction.outcome_path, extraction.be_moved) - pl_t = pl_theoretical(extraction.max_reached) - zi_ro = ZI_RO_MAP[day_short] - - return { - "id": str(row_id), - "screenshot_file": extraction.screenshot_file, - "source": source, - "data": extraction.data, - "zi": zi_ro, - "ora_ro": t_ro.strftime("%H:%M"), - "ora_utc": extraction.ora_utc, - "instrument": extraction.instrument, - "directie": extraction.directie, - "tf_mare": extraction.tf_mare, - "tf_mic": extraction.tf_mic, - "calitate": extraction.calitate, - "entry": f"{extraction.entry}", - "sl": f"{extraction.sl}", - "tp0": f"{extraction.tp0}", - "tp1": f"{extraction.tp1}", - "tp2": f"{extraction.tp2}", - "risc_pct": f"{extraction.risc_pct}", - "outcome_path": extraction.outcome_path, - "max_reached": extraction.max_reached, - "be_moved": str(extraction.be_moved), - "pl_marius": _format_optional(pl_m), - "pl_theoretical": _format_optional(pl_t), - "set": set_label, - "indicator_version": str(meta["indicator_version"]), - "pl_overlay_version": str(meta["pl_overlay_version"]), - "csv_schema_version": str(meta["csv_schema_version"]), - "extracted_at": extracted_at, - "note": extraction.note, - } - - -def _reject(reason: str) -> dict[str, Any]: - return {"status": "rejected", "reason": reason, "id": None, "row": None} - - -# --------------------------------------------------------------------------- -# public API -# --------------------------------------------------------------------------- - - -def append_extraction( - json_path: Path | str, - source: str, - csv_path: Path | str = "data/jurnal.csv", - meta_path: Path | str = "data/_meta.yaml", - calendar_path: Path | str = "calendar_evenimente.yaml", -) -> dict[str, Any]: - """Append one validated extraction to the jurnal CSV. - - Never raises. Returns one of: - - - ``{"status": "ok", "reason": "", "id": , "row": }`` - - ``{"status": "rejected", "reason": , "id": None, "row": None}`` - """ - json_path = Path(json_path) - csv_path = Path(csv_path) - meta_path = Path(meta_path) - calendar_path = Path(calendar_path) - - if source not in VALID_SOURCES: - return _reject( - f"invalid source {source!r}; must be one of {sorted(VALID_SOURCES)}" - ) - - if not json_path.exists(): - return _reject(f"JSON file not found: {json_path}") - - try: - with json_path.open("r", encoding="utf-8") as fh: - raw = fh.read() - except OSError as exc: - return _reject(f"failed to read JSON {json_path}: {exc}") - - try: - extraction = parse_extraction(raw) - except ValidationError as exc: - return _reject(f"validation error: {exc}") - except (ValueError, json.JSONDecodeError) as exc: - return _reject(f"validation error (json parse): {exc}") - - try: - meta = _load_meta(meta_path) - except (FileNotFoundError, OSError) as exc: - return _reject(f"_meta.yaml not found: {exc}") - except (ValueError, yaml.YAMLError) as exc: - return _reject(f"_meta.yaml invalid: {exc}") - - try: - calendar = load_calendar(calendar_path) - except (FileNotFoundError, OSError) as exc: - return _reject(f"calendar not found: {exc}") - except (ValueError, yaml.YAMLError) as exc: - return _reject(f"calendar invalid: {exc}") - - try: - existing = _read_existing_rows(csv_path) - except OSError as exc: - return _reject(f"failed to read existing CSV {csv_path}: {exc}") - - key = (extraction.screenshot_file, source) - for r in existing: - if (r.get("screenshot_file"), r.get("source")) == key: - return _reject( - f"duplicate row: screenshot_file={key[0]!r} source={key[1]!r}" - ) - - row_id = _next_id(existing) - extracted_at = ( - datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S") + "Z" - ) - - try: - row = _build_row( - extraction, - source=source, - row_id=row_id, - meta=meta, - calendar=calendar, - extracted_at=extracted_at, - ) - except (KeyError, ValueError) as exc: - return _reject(f"derived-field computation failed: {exc}") - - try: - _write_csv_atomic(csv_path, [*existing, row], list(CSV_COLUMNS)) - except OSError as exc: - return _reject( - f"atomic write failed: {exc}\n{traceback.format_exc()}" - ) - - return {"status": "ok", "reason": "", "id": row_id, "row": row} diff --git a/scripts/calendar_parse.py b/scripts/calendar_parse.py deleted file mode 100644 index 6680636..0000000 --- a/scripts/calendar_parse.py +++ /dev/null @@ -1,181 +0,0 @@ -"""Calendar parsing + Set classification for M2D backtesting. - -Each trade is tagged with a ``Set`` derived from its date, RO-local time, and the -economic-event calendar: - -- ``A1``: 16:35-17:00 RO, Tue/Wed/Thu -- ``A2``: 17:00-18:00 RO, Tue/Wed/Thu (sweet spot) -- ``A3``: 18:00-19:00 RO, Tue/Wed/Thu -- ``B`` : 22:00-22:45 RO, Tue/Wed/Thu -- ``C`` : inside the window of an event with severity in {extrem, mare} -- ``D`` : Mon or Fri -- ``Other``: anything else - -Priority: C > D > A1/A2/A3/B > Other. -""" - -from __future__ import annotations - -from datetime import date, datetime, time -from pathlib import Path -from typing import Any - -import yaml -from zoneinfo import ZoneInfo - -__all__ = [ - "RO_TZ", - "UTC_TZ", - "utc_to_ro", - "load_calendar", - "is_in_news_window", - "calc_set", -] - - -RO_TZ = ZoneInfo("Europe/Bucharest") -UTC_TZ = ZoneInfo("UTC") - -_DAY_SHORT = ("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun") - -_HIGH_SEVERITY = frozenset({"extrem", "mare"}) - -_WEEKLY_DAY_MAP = { - "monday": 0, - "tuesday": 1, - "wednesday": 2, - "thursday": 3, - "friday": 4, - "saturday": 5, - "sunday": 6, -} - - -def utc_to_ro(date_str: str, ora_utc_str: str) -> tuple[date, time, str]: - """Convert ``(YYYY-MM-DD, HH:MM UTC)`` to ``(date_ro, time_ro, day_short)``. - - DST-aware via :mod:`zoneinfo`. ``day_short`` is one of - ``Mon Tue Wed Thu Fri Sat Sun``. - """ - dt_utc = datetime.strptime(f"{date_str} {ora_utc_str}", "%Y-%m-%d %H:%M").replace( - tzinfo=UTC_TZ - ) - dt_ro = dt_utc.astimezone(RO_TZ) - return dt_ro.date(), dt_ro.time().replace(second=0, microsecond=0), _DAY_SHORT[dt_ro.weekday()] - - -def load_calendar(path: Path | str = "calendar_evenimente.yaml") -> list[dict[str, Any]]: - """Load a YAML calendar file. - - Validates ``schema_version == 1`` and returns the list of event dicts under - the top-level ``events`` key. - """ - p = Path(path) - with p.open("r", encoding="utf-8") as fh: - doc = yaml.safe_load(fh) - if not isinstance(doc, dict): - raise ValueError(f"calendar file {p} is not a mapping") - version = doc.get("schema_version") - if version != 1: - raise ValueError( - f"unsupported calendar schema_version: {version!r} (expected 1)" - ) - events = doc.get("events") or [] - if not isinstance(events, list): - raise ValueError(f"calendar events must be a list, got {type(events).__name__}") - return events - - -def _minutes(t: time) -> int: - return t.hour * 60 + t.minute - - -def _parse_hhmm(s: str) -> time: - return datetime.strptime(s, "%H:%M").time() - - -def _is_first_friday_of_month(d: date) -> bool: - return d.weekday() == 4 and d.day <= 7 - - -def _event_matches_date(event: dict[str, Any], d: date) -> bool: - cadence = event.get("cadence", "") - if cadence == "scheduled": - ev_date_raw = event.get("date") - if isinstance(ev_date_raw, date): - ev_date = ev_date_raw - elif isinstance(ev_date_raw, str): - ev_date = datetime.strptime(ev_date_raw, "%Y-%m-%d").date() - else: - return False - return ev_date == d - if cadence == "first_friday_monthly": - return _is_first_friday_of_month(d) - if cadence.startswith("weekly_"): - day_name = cadence[len("weekly_") :].lower() - target = _WEEKLY_DAY_MAP.get(day_name) - if target is None: - return False - return d.weekday() == target - # cadences below are not pinned down to a precise calendar day yet, so we - # do not trigger Set C for them. ADP pre-NFP is also explicitly deferred. - return False - - -def is_in_news_window(d: date, t: time, calendar: list[dict[str, Any]]) -> bool: - """Return True iff ``(d, t)`` falls inside the window of a high-severity event. - - Window: ``[time_ro - window_before_min, time_ro + window_after_min]`` (inclusive - on both ends). Only events with ``severity`` in ``{extrem, mare}`` count. - - Cadences honoured: ``scheduled``, ``first_friday_monthly``, ``weekly_``. - Other cadences (``monthly_mid``, ``monthly_end``, ``monthly_15``, - ``wednesday_pre_nfp``, ``monthly_first_week`` etc.) are deferred and never - trigger Set C. - """ - t_min = _minutes(t) - for event in calendar: - if event.get("severity") not in _HIGH_SEVERITY: - continue - if not _event_matches_date(event, d): - continue - ev_time_raw = event.get("time_ro") - if isinstance(ev_time_raw, time): - ev_time = ev_time_raw - elif isinstance(ev_time_raw, str): - ev_time = _parse_hhmm(ev_time_raw) - else: - continue - center = _minutes(ev_time) - before = int(event.get("window_before_min", 0)) - after = int(event.get("window_after_min", 0)) - if center - before <= t_min <= center + after: - return True - return False - - -def _in_range(t: time, lo: time, hi: time) -> bool: - """Half-open ``[lo, hi)`` containment.""" - return _minutes(lo) <= _minutes(t) < _minutes(hi) - - -def calc_set(d: date, t: time, day_of_week: str, calendar: list[dict[str, Any]]) -> str: - """Classify a trade into one of ``A1 A2 A3 B C D Other``. - - Priority: ``C`` (news) > ``D`` (Mon/Fri) > ``A1/A2/A3/B`` (time bands on - Tue/Wed/Thu) > ``Other``. - """ - if is_in_news_window(d, t, calendar): - return "C" - if day_of_week in ("Mon", "Fri"): - return "D" - if day_of_week in ("Tue", "Wed", "Thu"): - if _in_range(t, time(16, 35), time(17, 0)): - return "A1" - if _in_range(t, time(17, 0), time(18, 0)): - return "A2" - if _in_range(t, time(18, 0), time(19, 0)): - return "A3" - if _in_range(t, time(22, 0), time(22, 45)): - return "B" - return "Other" diff --git a/scripts/generate_template.py b/scripts/generate_template.py new file mode 100644 index 0000000..e513cb6 --- /dev/null +++ b/scripts/generate_template.py @@ -0,0 +1,731 @@ +"""Generator pentru data/backtest.xlsx. + +5 strategii de management comparate side-by-side pe semnale blackbox: + - TP0 only : 100% close la TP0 + - TP1 only : 100% OCO la SL/TP1 + - TP2 only : 100% OCO la SL/TP2 + - Hybrid + BE : 50% TP0 + mut SL la BE + 50% TP1 (recomandat de trader) + - Hybrid no BE : 50% TP0 + 50% TP1, fără BE (control pentru a izola valoarea BE-ului) + +Rulare: + pip install openpyxl + python scripts/generate_template.py +""" + +from __future__ import annotations + +from datetime import date, time +from pathlib import Path + +from openpyxl import Workbook +from openpyxl.chart import LineChart, Reference +from openpyxl.formatting.rule import CellIsRule +from openpyxl.styles import Alignment, Border, Font, PatternFill, Side +from openpyxl.utils import get_column_letter +from openpyxl.worksheet.datavalidation import DataValidation + + +OUTPUT = Path(__file__).resolve().parent.parent / "data" / "backtest.xlsx" +MAX_ROWS = 500 # rânduri pre-completate cu formule în sheet-ul Trades + +# --------------------------------------------------------------------------- +# Styles +# --------------------------------------------------------------------------- + +HEADER_FILL = PatternFill("solid", fgColor="1F3864") +HEADER_FONT = Font(name="Calibri", size=11, bold=True, color="FFFFFF") +INPUT_FILL = PatternFill("solid", fgColor="FFF8E1") +DERIVED_FILL = PatternFill("solid", fgColor="E8F1FA") +HIDDEN_FILL = PatternFill("solid", fgColor="F0F0F0") +TITLE_FONT = Font(name="Calibri", size=16, bold=True, color="1F3864") +SUBTITLE_FONT = Font(name="Calibri", size=12, bold=True, color="1F3864") +THIN = Side(border_style="thin", color="BFBFBF") +BORDER = Border(left=THIN, right=THIN, top=THIN, bottom=THIN) +CENTER = Alignment(horizontal="center", vertical="center") +LEFT = Alignment(horizontal="left", vertical="center") +RIGHT = Alignment(horizontal="right", vertical="center") + + +# --------------------------------------------------------------------------- +# Lists +# --------------------------------------------------------------------------- + +STRATEGIES = ["M2D", "EMA cross", "Order block", "Liquidity sweep", "Custom"] +SESSIONS = ["A1", "A2", "A3", "B", "C", "D", "Other"] +INDICATORS = ["DIA", "US30", "SPY", "QQQ", "ES", "NQ"] +TIMEFRAMES = ["1min", "3min", "15min"] +DIRECTIONS = ["Buy", "Sell"] +OUTCOMES = ["SL", "TP0 only", "TP1", "TP2"] + +# Cele 5 strategii de management (sufix folosit în numele coloanelor) + label friendly +STRAT_KEYS = ["tp0only", "tp1only", "tp2only", "hybrid_be", "hybrid_nobe"] +STRAT_LABELS = { + "tp0only": "TP0 only", + "tp1only": "TP1 only", + "tp2only": "TP2 only", + "hybrid_be": "Hybrid + BE", + "hybrid_nobe": "Hybrid no BE", +} + +# --------------------------------------------------------------------------- +# Trades sheet — schema +# --------------------------------------------------------------------------- + +INPUT_HEADERS = [ + "#", "Data", "Ora RO", "Zi", "Sesiune", + "Strategie", "Indicator", "TF", + "Direcție", "SL %", "TP0 %", "TP1 %", "TP2 %", + "Outcome", "Notes", +] +DERIVED_HEADERS = ( + [f"R_{s}" for s in STRAT_KEYS] + + [f"$_{s}" for s in STRAT_KEYS] + + [f"Bal_{s}" for s in STRAT_KEYS] +) +HELPER_HEADERS = ( + [f"Win_{s}" for s in STRAT_KEYS] + + [f"Peak_{s}" for s in STRAT_KEYS] + + [f"DD_{s}" for s in STRAT_KEYS] +) +TRADES_HEADERS = INPUT_HEADERS + DERIVED_HEADERS + HELPER_HEADERS + +# Mapă nume → literă coloană Excel +COL = {name: get_column_letter(i + 1) for i, name in enumerate(TRADES_HEADERS)} + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _col_to_int(letter: str) -> int: + n = 0 + for ch in letter: + n = n * 26 + (ord(ch) - ord("A") + 1) + return n + + +# --------------------------------------------------------------------------- +# Config sheet +# --------------------------------------------------------------------------- + + +def build_config(wb: Workbook) -> None: + ws = wb.create_sheet("Config", 0) + ws.sheet_view.showGridLines = False + + ws["A1"] = "📋 Config — editează doar celulele galbene" + ws["A1"].font = TITLE_FONT + ws.merge_cells("A1:C1") + + ws["A3"] = "Setting" + ws["B3"] = "Value" + ws["C3"] = "Note" + for c in ("A3", "B3", "C3"): + ws[c].font = HEADER_FONT + ws[c].fill = HEADER_FILL + ws[c].alignment = CENTER + + ws["A4"] = "Account Size Start ($)" + ws["B4"] = 10000 + ws["C4"] = "Balanța inițială pentru calcule $ și HWM" + + ws["A5"] = "Risk per Trade (%)" + ws["B5"] = 1.0 + ws["C5"] = "% din account riscat per trade (= -1R)" + + ws["A6"] = "Risk per Trade ($)" + ws["B6"] = "=B4*B5/100" + ws["C6"] = "Auto — derivat din B4 și B5" + + for r in (4, 5): + ws.cell(row=r, column=2).fill = INPUT_FILL + ws.cell(row=r, column=2).border = BORDER + ws["B6"].fill = DERIVED_FILL + ws["B6"].border = BORDER + ws["B4"].number_format = "$#,##0" + ws["B5"].number_format = '0.0"%"' + ws["B6"].number_format = "$#,##0.00" + + # Liste dropdown — coloanele E–J (6 coloane) + list_columns = [ + ("Strategii", STRATEGIES), + ("Sesiuni (auto)", SESSIONS), + ("Indicatori", INDICATORS), + ("TF", TIMEFRAMES), + ("Direcție", DIRECTIONS), + ("Outcome", OUTCOMES), + ] + for col_idx, (label, values) in enumerate(list_columns, start=5): + cell = ws.cell(row=3, column=col_idx, value=label) + cell.font = HEADER_FONT + cell.fill = HEADER_FILL + cell.alignment = CENTER + for row_idx, v in enumerate(values, start=4): + c = ws.cell(row=row_idx, column=col_idx, value=v) + c.alignment = CENTER + + widths = { + "A": 24, "B": 14, "C": 38, "D": 2, + "E": 14, "F": 14, "G": 13, "H": 10, "I": 10, "J": 12, + } + for col, w in widths.items(): + ws.column_dimensions[col].width = w + + +# --------------------------------------------------------------------------- +# Formula builders pentru Trades sheet +# --------------------------------------------------------------------------- + + +def _f_day(r: int) -> str: + d = f'{COL["Data"]}{r}' + return ( + f'=IF({d}="","",' + f'CHOOSE(WEEKDAY({d},2),"Lu","Ma","Mi","Jo","Vi","Sa","Du"))' + ) + + +def _f_session(r: int) -> str: + """Derivă Sesiunea M2D din Data + Ora RO.""" + d = f'{COL["Data"]}{r}' + t = f'{COL["Ora RO"]}{r}' + wd = f"WEEKDAY({d},2)" + mid_week = f"AND({wd}>=2,{wd}<=4)" + return ( + f'=IF(OR({d}="",{t}=""),"",' + f"IF(OR({wd}=1,{wd}=5),\"D\"," + f'IF(AND({t}>=TIME(15,30,0),{t}=TIME(16,35,0),{t}=TIME(17,0,0),{t}=TIME(18,0,0),{t}=TIME(22,0,0),{t} str: + o = f'{COL["Outcome"]}{r}' + sl = f'{COL["SL %"]}{r}' + tp0 = f'{COL["TP0 %"]}{r}' + return f'=IF({o}="","",IF({o}="SL",-1,{tp0}/{sl}))' + + +def _f_r_tp1only(r: int) -> str: + o = f'{COL["Outcome"]}{r}' + sl = f'{COL["SL %"]}{r}' + tp1 = f'{COL["TP1 %"]}{r}' + return ( + f'=IF({o}="","",' + f'IF(OR({o}="SL",{o}="TP0 only"),-1,{tp1}/{sl}))' + ) + + +def _f_r_tp2only(r: int) -> str: + o = f'{COL["Outcome"]}{r}' + sl = f'{COL["SL %"]}{r}' + tp2 = f'{COL["TP2 %"]}{r}' + return f'=IF({o}="","",IF({o}="TP2",{tp2}/{sl},-1))' + + +def _f_r_hybrid_be(r: int) -> str: + o = f'{COL["Outcome"]}{r}' + sl = f'{COL["SL %"]}{r}' + tp0 = f'{COL["TP0 %"]}{r}' + tp1 = f'{COL["TP1 %"]}{r}' + return ( + f'=IF({o}="","",' + f'IF({o}="SL",-1,' + f'IF({o}="TP0 only",0.5*{tp0}/{sl},' + f'0.5*({tp0}+{tp1})/{sl})))' + ) + + +def _f_r_hybrid_nobe(r: int) -> str: + o = f'{COL["Outcome"]}{r}' + sl = f'{COL["SL %"]}{r}' + tp0 = f'{COL["TP0 %"]}{r}' + tp1 = f'{COL["TP1 %"]}{r}' + return ( + f'=IF({o}="","",' + f'IF({o}="SL",-1,' + f'IF({o}="TP0 only",0.5*{tp0}/{sl}-0.5,' + f'0.5*({tp0}+{tp1})/{sl})))' + ) + + +R_FN: dict[str, callable] = { + "tp0only": _f_r_tp0only, + "tp1only": _f_r_tp1only, + "tp2only": _f_r_tp2only, + "hybrid_be": _f_r_hybrid_be, + "hybrid_nobe": _f_r_hybrid_nobe, +} + + +def _f_dollar(r: int, r_col: str) -> str: + rc = f"{COL[r_col]}{r}" + return f'=IF({rc}="","",{rc}*Config!$B$6)' + + +def _f_balance(r: int, dollar_col: str) -> str: + dc = COL[dollar_col] + return f'=IF({dc}{r}="","",Config!$B$4 + SUM(${dc}$2:{dc}{r}))' + + +def _f_win(r: int, r_col: str) -> str: + rc = f"{COL[r_col]}{r}" + return f'=IF({rc}="","",IF({rc}>0,1,0))' + + +def _f_peak(r: int, balance_col: str, peak_col: str) -> str: + bc = COL[balance_col] + pc = COL[peak_col] + if r == 2: + return f'=IF({bc}{r}="","",{bc}{r})' + return ( + f'=IF({bc}{r}="","",' + f'IF({pc}{r-1}="",{bc}{r},MAX({pc}{r-1},{bc}{r})))' + ) + + +def _f_drawdown(r: int, peak_col: str, balance_col: str) -> str: + pc = f"{COL[peak_col]}{r}" + bc = f"{COL[balance_col]}{r}" + return f'=IF({bc}="","",{pc}-{bc})' + + +# --------------------------------------------------------------------------- +# Trades sheet +# --------------------------------------------------------------------------- + + +def build_trades(wb: Workbook) -> None: + ws = wb.create_sheet("Trades", 1) + ws.sheet_view.showGridLines = False + ws.freeze_panes = "B2" + + # Headers + for col_idx, header in enumerate(TRADES_HEADERS, start=1): + cell = ws.cell(row=1, column=col_idx, value=header) + cell.font = HEADER_FONT + cell.fill = HEADER_FILL + cell.alignment = CENTER + cell.border = BORDER + + # Formule pe toate rândurile pre-pregătite + for r in range(2, MAX_ROWS + 2): + ws.cell(row=r, column=1, value="=ROW()-1") + ws[f'{COL["Zi"]}{r}'] = _f_day(r) + ws[f'{COL["Sesiune"]}{r}'] = _f_session(r) + + for strat in STRAT_KEYS: + ws[f'{COL[f"R_{strat}"]}{r}'] = R_FN[strat](r) + ws[f'{COL[f"$_{strat}"]}{r}'] = _f_dollar(r, f"R_{strat}") + ws[f'{COL[f"Bal_{strat}"]}{r}'] = _f_balance(r, f"$_{strat}") + ws[f'{COL[f"Win_{strat}"]}{r}'] = _f_win(r, f"R_{strat}") + ws[f'{COL[f"Peak_{strat}"]}{r}'] = _f_peak( + r, f"Bal_{strat}", f"Peak_{strat}" + ) + ws[f'{COL[f"DD_{strat}"]}{r}'] = _f_drawdown( + r, f"Peak_{strat}", f"Bal_{strat}" + ) + + # Sample row 2 + ws["B2"] = date(2026, 5, 13) + ws["C2"] = time(17, 33) + ws[f'{COL["Strategie"]}2'] = "M2D" + ws[f'{COL["Indicator"]}2'] = "DIA" + ws[f'{COL["TF"]}2'] = "1min" + ws[f'{COL["Direcție"]}2'] = "Sell" + ws[f'{COL["SL %"]}2'] = 0.30 + ws[f'{COL["TP0 %"]}2'] = 0.10 + ws[f'{COL["TP1 %"]}2'] = 0.15 + ws[f'{COL["TP2 %"]}2'] = 0.30 + ws[f'{COL["Outcome"]}2'] = "TP1" + ws[f'{COL["Notes"]}2'] = "Exemplu — șterge când începi" + + # Number formats + for col_name in ("SL %", "TP0 %", "TP1 %", "TP2 %"): + for r in range(2, MAX_ROWS + 2): + ws[f"{COL[col_name]}{r}"].number_format = '0.000"%"' + + for strat in STRAT_KEYS: + for r in range(2, MAX_ROWS + 2): + ws[f"{COL[f'R_{strat}']}{r}"].number_format = "+0.000;-0.000;0.000" + for prefix in ("$_", "Bal_", "Peak_", "DD_"): + ws[f"{COL[f'{prefix}{strat}']}{r}"].number_format = '"$"#,##0.00' + + for r in range(2, MAX_ROWS + 2): + ws[f"B{r}"].number_format = "yyyy-mm-dd" + + # Coloring + input_letters = { + COL[n] + for n in ( + "Data", "Ora RO", "Strategie", "Indicator", "TF", + "Direcție", "SL %", "TP0 %", "TP1 %", "TP2 %", + "Outcome", "Notes", + ) + } + derived_letters = {COL["Zi"], COL["Sesiune"]} + for strat in STRAT_KEYS: + derived_letters.add(COL[f"R_{strat}"]) + derived_letters.add(COL[f"$_{strat}"]) + derived_letters.add(COL[f"Bal_{strat}"]) + helper_letters = set() + for strat in STRAT_KEYS: + for prefix in ("Win_", "Peak_", "DD_"): + helper_letters.add(COL[f"{prefix}{strat}"]) + + for r in range(2, MAX_ROWS + 2): + for cl in input_letters: + ws[f"{cl}{r}"].fill = INPUT_FILL + for cl in derived_letters: + ws[f"{cl}{r}"].fill = DERIVED_FILL + for cl in helper_letters: + ws[f"{cl}{r}"].fill = HIDDEN_FILL + + # Column widths + widths = { + "A": 5, "B": 12, "C": 9, "D": 5, "E": 9, + "F": 12, "G": 11, "H": 8, "I": 9, + "J": 9, "K": 9, "L": 9, "M": 9, + "N": 11, "O": 28, + } + for col, w in widths.items(): + ws.column_dimensions[col].width = w + # Derived + helper: width 11 + for strat in STRAT_KEYS: + for prefix in ("R_", "$_", "Bal_", "Win_", "Peak_", "DD_"): + ws.column_dimensions[COL[f"{prefix}{strat}"]].width = 11 + + # Data validation dropdowns + def _add_dv(col_name: str, source: str) -> None: + cl = COL[col_name] + dv = DataValidation( + type="list", formula1=source, + allow_blank=True, showErrorMessage=True, + ) + dv.error = "Valoare invalidă — folosește dropdown-ul." + dv.errorTitle = "Input invalid" + dv.add(f"{cl}2:{cl}{MAX_ROWS + 1}") + ws.add_data_validation(dv) + + # Config columns: E=Strategii, F=Sesiuni, G=Indicatori, H=TF, I=Direcție, J=Outcome + _add_dv("Strategie", "=Config!$E$4:$E$8") + _add_dv("Indicator", "=Config!$G$4:$G$9") + _add_dv("TF", "=Config!$H$4:$H$6") + _add_dv("Direcție", "=Config!$I$4:$I$5") + _add_dv("Outcome", "=Config!$J$4:$J$7") + + # Conditional formatting pe coloanele R (5 strategii) + green_fill = PatternFill("solid", fgColor="C6EFCE") + red_fill = PatternFill("solid", fgColor="FFC7CE") + grey_fill = PatternFill("solid", fgColor="D9D9D9") + for strat in STRAT_KEYS: + cl = COL[f"R_{strat}"] + rng = f"{cl}2:{cl}{MAX_ROWS + 1}" + ws.conditional_formatting.add( + rng, CellIsRule(operator="greaterThan", formula=["0"], fill=green_fill) + ) + ws.conditional_formatting.add( + rng, CellIsRule(operator="lessThan", formula=["0"], fill=red_fill) + ) + ws.conditional_formatting.add( + rng, CellIsRule(operator="equal", formula=["0"], fill=grey_fill) + ) + + +# --------------------------------------------------------------------------- +# Dashboard sheet +# --------------------------------------------------------------------------- + + +def _range(col_name: str) -> str: + cl = COL[col_name] + return f"Trades!${cl}$2:${cl}${MAX_ROWS + 1}" + + +METRIC_HINTS: dict[str, str] = { + "Trades Placed": "Numărul total de trade-uri logate", + "Wins": "Trade-uri cu R > 0", + "Win Ratio": "% wins. Singur NU spune mult — vezi împreună cu R:R și Expectancy", + "Average Win ($)": "Câștigul mediu pe trade winning", + "Average Loss ($)": "Pierderea medie pe trade losing", + "Best Trade ($)": "Cel mai mare câștig individual", + "Worst Trade ($)": "Cea mai mare pierdere individuală", + "Profit Factor": ">1.0 profitabil • >1.5 solid • >2.0 foarte bun • <1.0 pierzător", + "Risk:Reward": "Avg Win ÷ |Avg Loss|. >1 = câștig mediu > pierdere medie", + "Expectancy (R)": "★ STEAUA NORDULUI ★ >+0.20R = GO LIVE • negativ = ABANDON", + "Expectancy ($)": "Expectancy R convertit în $ (folosește Risk per Trade)", + "Cumulative P&L ($)": "P&L total în $ pe toate trade-urile", + "HWM Balance ($)": "Highest watermark — balanța de vârf atinsă", + "Max Drawdown ($)": "Cea mai mare cădere ($) din vârf la fund", +} + + +def build_dashboard(wb: Workbook) -> None: + ws = wb.create_sheet("Dashboard", 2) + ws.sheet_view.showGridLines = False + + ws["A1"] = "📊 Backtest Dashboard" + ws["A1"].font = TITLE_FONT + ws.merge_cells("A1:G1") + + ws["A2"] = ( + "Comparație 5 strategii management — pe aceleași semnale blackbox" + ) + ws["A2"].font = Font(name="Calibri", size=10, italic=True, color="595959") + ws.merge_cells("A2:G2") + + # Row 4: headers (5 columns B-F pentru strategii + G pentru "Cum citesc") + ws["A4"] = "Metric" + strat_cols = {} # strat_key → column letter (B/C/D/E/F) + for i, strat in enumerate(STRAT_KEYS): + letter = get_column_letter(2 + i) + strat_cols[strat] = letter + ws[f"{letter}4"] = STRAT_LABELS[strat] + ws["G4"] = "Cum citesc" + for letter in ["A"] + list(strat_cols.values()) + ["G"]: + c = ws[f"{letter}4"] + c.font = HEADER_FONT + c.fill = HEADER_FILL + c.alignment = CENTER + c.border = BORDER + + # Ranges per strategie + R = {s: _range(f"R_{s}") for s in STRAT_KEYS} + D = {s: _range(f"$_{s}") for s in STRAT_KEYS} + W = {s: _range(f"Win_{s}") for s in STRAT_KEYS} + BAL = {s: _range(f"Bal_{s}") for s in STRAT_KEYS} + DD = {s: _range(f"DD_{s}") for s in STRAT_KEYS} + OUTCOME_RANGE = _range("Outcome") + + # Metric rows — fiecare metric e un dict cu per-strategy formula + format + metrics: list[tuple[str, callable, str]] = [ + # (label, fn(strat_key) -> formula, number_format) + ("Trades Placed", lambda s: f'=COUNTA({OUTCOME_RANGE})', "0"), + ("Wins", lambda s: f'=COUNTIF({W[s]},1)', "0"), + # Win Ratio: depends on rows above — handled after metrics list (placeholder) + ("Win Ratio", lambda s: None, "0.0%"), + ("Average Win ($)", lambda s: f'=IFERROR(AVERAGEIF({D[s]},">0"),0)', '"$"#,##0.00'), + ("Average Loss ($)", lambda s: f'=IFERROR(AVERAGEIF({D[s]},"<0"),0)', '"$"#,##0.00'), + ("Best Trade ($)", lambda s: f'=IFERROR(MAX({D[s]}),0)', '"$"#,##0.00'), + ("Worst Trade ($)", lambda s: f'=IFERROR(MIN({D[s]}),0)', '"$"#,##0.00'), + ("Profit Factor", lambda s: f'=IFERROR(SUMIF({D[s]},">0")/ABS(SUMIF({D[s]},"<0")),0)', "0.00"), + # Risk:Reward — placeholder; bazat pe rândurile Avg Win/Loss + ("Risk:Reward", lambda s: None, "0.00"), + ("Expectancy (R)", lambda s: f'=IFERROR(AVERAGE({R[s]}),0)', "+0.000;-0.000;0.000"), + ("Expectancy ($)", lambda s: f'=IFERROR(AVERAGE({D[s]}),0)', '"$"#,##0.00'), + ("Cumulative P&L ($)", lambda s: f'=SUM({D[s]})', '"$"#,##0.00'), + # HWM — placeholder cu ref la Trades Placed (row 5) + ("HWM Balance ($)", lambda s: None, '"$"#,##0.00'), + ("Max Drawdown ($)", lambda s: f'=IFERROR(MAX({DD[s]}),0)', '"$"#,##0.00'), + ] + + # Determine row indexes pentru formule speciale (depind de poziție) + label_to_row = {label: 5 + idx for idx, (label, _, _) in enumerate(metrics)} + trades_row = label_to_row["Trades Placed"] + wins_row = label_to_row["Wins"] + avg_win_row = label_to_row["Average Win ($)"] + avg_loss_row = label_to_row["Average Loss ($)"] + + for idx, (label, fn, fmt) in enumerate(metrics): + r = 5 + idx + ws[f"A{r}"] = label + ws[f"A{r}"].font = Font(name="Calibri", size=11, bold=True) + ws[f"A{r}"].border = BORDER + ws[f"A{r}"].alignment = LEFT + for strat in STRAT_KEYS: + letter = strat_cols[strat] + if label == "Win Ratio": + formula = f"=IFERROR({letter}{wins_row}/{letter}{trades_row},0)" + elif label == "Risk:Reward": + formula = f"=IFERROR({letter}{avg_win_row}/ABS({letter}{avg_loss_row}),0)" + elif label == "HWM Balance ($)": + formula = ( + f"=IF({letter}{trades_row}=0,Config!$B$4,MAX({BAL[strat]}))" + ) + else: + formula = fn(strat) + cell = ws[f"{letter}{r}"] + cell.value = formula + cell.number_format = fmt + cell.fill = DERIVED_FILL + cell.border = BORDER + cell.alignment = RIGHT + # Coloana G — interpretare scurtă + hint_cell = ws[f"G{r}"] + hint_cell.value = METRIC_HINTS.get(label, "") + hint_cell.font = Font(name="Calibri", size=10, italic=True, color="595959") + hint_cell.alignment = Alignment(horizontal="left", vertical="center", wrap_text=True) + hint_cell.border = BORDER + + # ---- Glosar section: exemple concrete pentru metricile-cheie ---- + glosar_start = 5 + len(metrics) + 2 # 2 rânduri spațiu după metrici + ws[f"A{glosar_start}"] = "📖 Glosar metrici — exemple concrete" + ws[f"A{glosar_start}"].font = SUBTITLE_FONT + ws.merge_cells(f"A{glosar_start}:G{glosar_start}") + + glosar_entries = [ + ( + "Profit Factor", + "Suma câștigurilor ÷ |suma pierderilor|. Total cumulativ, nu mediu.", + "10 trade-uri: 4 wins de $50 (=$200) + 6 losses de −$30 (=−$180). PF = 200÷180 = 1.11 (marginal profitabil). La PF=2.0 câștigi de 2× cât pierzi în total.", + ), + ( + "Risk:Reward", + "Avg Win ÷ |Avg Loss|. Privește per-trade, nu total.", + "Avg win $50, avg loss −$30 → R:R = 1.67. La R:R=2.0 ești profitabil chiar cu Win Ratio doar 40%. La R:R=0.5 ai nevoie de WR >67%.", + ), + ( + "Expectancy (R)", + "Câștigul mediu per trade exprimat în multipli de risc (R). CEA MAI ONESTĂ metrică — combină WR și R:R într-un singur număr.", + "10 trade-uri cu R = [+0.5, +0.5, +0.5, +0.5, −1, −1, −1, −1, −1, −1] → media = −0.30R (pierdere) chiar dacă WR=40%. Pragul GO LIVE din STOPPING_RULE.md: ≥ +0.20R.", + ), + ( + "Win Ratio (WR)", + "% trade-uri cu R > 0. ÎNȘELĂTOR singur — un WR mare cu R:R mic poate fi pierzător.", + "WR=70% pare excelent, dar dacă R:R=0.3 (câștigi $30, pierzi $100) → Expectancy = 0.7·30 − 0.3·100 = −$9 per trade. Pierzător.", + ), + ( + "Max Drawdown", + "Cea mai mare cădere din vârful balanței la fundul ulterior. Măsoară 'durerea psihologică'.", + "Balance peak $11,500 → fund $9,800 → DD = $1,700 (17% din peak). DD mare la backtest = greu de tolerat în live.", + ), + ] + + row = glosar_start + 1 + for term, definition, example in glosar_entries: + ws[f"A{row}"] = term + ws[f"A{row}"].font = Font(name="Calibri", size=11, bold=True, color="1F3864") + ws[f"A{row}"].alignment = Alignment(horizontal="left", vertical="top", wrap_text=True) + ws[f"B{row}"] = definition + ws[f"B{row}"].font = Font(name="Calibri", size=10) + ws[f"B{row}"].alignment = Alignment(horizontal="left", vertical="top", wrap_text=True) + ws.merge_cells(f"B{row}:C{row}") + ws[f"D{row}"] = f"Exemplu: {example}" + ws[f"D{row}"].font = Font(name="Calibri", size=10, italic=True, color="595959") + ws[f"D{row}"].alignment = Alignment(horizontal="left", vertical="top", wrap_text=True) + ws.merge_cells(f"D{row}:G{row}") + ws.row_dimensions[row].height = 48 + row += 1 + + glosar_end = row # primul rând după glosar + + # Helper pentru a emite un block breakdown (per Sesiune / Strategie / etc.) + def _emit_breakdown( + start_row: int, title: str, first_col_label: str, + items: list[str], item_range: str, overlay_strat: str, + ) -> int: + ws[f"A{start_row}"] = title + ws[f"A{start_row}"].font = SUBTITLE_FONT + ws.merge_cells(f"A{start_row}:F{start_row}") + headers = [first_col_label, "N", "Wins", "WR", "Expectancy R", "Cum $"] + for col_idx, h in enumerate(headers, start=1): + c = ws.cell(row=start_row + 1, column=col_idx, value=h) + c.font = HEADER_FONT + c.fill = HEADER_FILL + c.alignment = CENTER + c.border = BORDER + for i, item in enumerate(items): + r = start_row + 2 + i + ws[f"A{r}"] = item + ws[f"B{r}"] = f'=COUNTIF({item_range},"{item}")' + ws[f"C{r}"] = f'=COUNTIFS({item_range},"{item}",{W[overlay_strat]},1)' + ws[f"D{r}"] = f"=IFERROR(C{r}/B{r},0)" + ws[f"E{r}"] = ( + f'=IFERROR(AVERAGEIFS({R[overlay_strat]},{item_range},"{item}"),0)' + ) + ws[f"F{r}"] = f'=SUMIFS({D[overlay_strat]},{item_range},"{item}")' + ws[f"B{r}"].number_format = "0" + ws[f"C{r}"].number_format = "0" + ws[f"D{r}"].number_format = "0.0%" + ws[f"E{r}"].number_format = "+0.000;-0.000;0.000" + ws[f"F{r}"].number_format = '"$"#,##0.00' + for c in ("A", "B", "C", "D", "E", "F"): + ws[f"{c}{r}"].border = BORDER + ws[f"{c}{r}"].alignment = RIGHT if c != "A" else LEFT + return start_row + 2 + len(items) + + # Breakdowns — toate folosesc overlay-ul Hybrid+BE (recomandat de trader) + overlay = "hybrid_be" + start = glosar_end + 2 # 2 rânduri spațiu după glosar + after_sess = _emit_breakdown( + start, "PER SESIUNE (overlay: Hybrid + BE)", "Sesiune", + SESSIONS, _range("Sesiune"), overlay, + ) + after_strat = _emit_breakdown( + after_sess + 2, "PER STRATEGIE (overlay: Hybrid + BE)", "Strategie", + STRATEGIES, _range("Strategie"), overlay, + ) + after_ind = _emit_breakdown( + after_strat + 2, "PER INDICATOR (overlay: Hybrid + BE)", "Indicator", + INDICATORS, _range("Indicator"), overlay, + ) + _emit_breakdown( + after_ind + 2, "PER DIRECȚIE (overlay: Hybrid + BE)", "Direcție", + DIRECTIONS, _range("Direcție"), overlay, + ) + + # Column widths + widths = {"A": 22, "B": 14, "C": 14, "D": 14, "E": 16, "F": 16, "G": 50} + for col, w in widths.items(): + ws.column_dimensions[col].width = w + + # Row height pentru rândurile cu hint (cu wrap) + for r in range(5, 5 + len(metrics)): + ws.row_dimensions[r].height = 22 + + # Equity curve chart — 5 linii + chart = LineChart() + chart.title = "Equity Curve — 5 strategii" + chart.style = 12 + chart.y_axis.title = "Balance ($)" + chart.x_axis.title = "Trade #" + chart.height = 12 + chart.width = 24 + + data = Reference( + wb["Trades"], + min_col=_col_to_int(COL[f"Bal_{STRAT_KEYS[0]}"]), + max_col=_col_to_int(COL[f"Bal_{STRAT_KEYS[-1]}"]), + min_row=1, + max_row=MAX_ROWS + 1, + ) + chart.add_data(data, titles_from_data=True) + cats = Reference( + wb["Trades"], min_col=1, max_col=1, + min_row=2, max_row=MAX_ROWS + 1, + ) + chart.set_categories(cats) + ws.add_chart(chart, "H4") + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + + +def build_workbook() -> Workbook: + wb = Workbook() + default = wb.active + wb.remove(default) + build_config(wb) + build_trades(wb) + build_dashboard(wb) + wb.active = wb.sheetnames.index("Dashboard") + return wb + + +def main() -> int: + OUTPUT.parent.mkdir(parents=True, exist_ok=True) + wb = build_workbook() + wb.save(OUTPUT) + print(f"Wrote {OUTPUT}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/manual_log.py b/scripts/manual_log.py deleted file mode 100644 index 04d2edd..0000000 --- a/scripts/manual_log.py +++ /dev/null @@ -1,134 +0,0 @@ -"""Helper for manual M2D trade entry — derives full M2DExtraction dict from minimal user inputs. - -User provides 6 required fields: data, ora_ro, directie, entry, sl, outcome_path. -All other fields default or are computed: - - tp0 = entry ± 0.4 × |entry - sl| - - tp1 = entry ± 0.6 × |entry - sl| - - tp2 = entry ± 1.0 × |entry - sl| (symmetric with sl) - - risc_pct = 100 × |entry - sl| / entry - - ora_utc = ora_ro converted via Europe/Bucharest (DST-aware) - - max_reached derived from outcome_path - - be_moved = True if outcome contains TP0 else False - - tf_mare/tf_mic default 5min/1min - - calitate default 'n/a' - - confidence = 'high' (manual entry) - - screenshot_file generated if not provided: --.png -""" - -from __future__ import annotations - -from datetime import date, datetime, time -from typing import Literal -from zoneinfo import ZoneInfo - -RO_TZ = ZoneInfo("Europe/Bucharest") -UTC_TZ = ZoneInfo("UTC") - - -OUTCOME_TO_MAX_REACHED = { - "SL": "SL_first", - "TP0→SL": "TP0", - "TP0→TP1": "TP1", - "TP0→TP2": "TP2", - "TP0→pending": "TP0", - "pending": "SL_first", # placeholder; user can override -} - -OUTCOME_TO_BE_MOVED = { - "SL": False, - "TP0→SL": True, # BE move should have happened; True = rule-enforced - "TP0→TP1": True, - "TP0→TP2": True, - "TP0→pending": True, - "pending": False, -} - - -def ro_to_utc(data_iso: str, ora_ro_str: str) -> str: - """Convert (YYYY-MM-DD, HH:MM RO) -> HH:MM UTC string, DST-aware.""" - d = date.fromisoformat(data_iso) - t = datetime.strptime(ora_ro_str, "%H:%M").time() - dt_ro = datetime.combine(d, t, tzinfo=RO_TZ) - dt_utc = dt_ro.astimezone(UTC_TZ) - return dt_utc.strftime("%H:%M") - - -def build_extraction( - data: str, - ora_ro: str, - directie: Literal["Buy", "Sell"], - entry: float, - sl: float, - outcome_path: Literal["SL", "TP0→SL", "TP0→TP1", "TP0→TP2", "TP0→pending", "pending"], - instrument: Literal["DIA", "US30", "other"] = "DIA", - tf_mare: Literal["5min", "15min"] = "5min", - tf_mic: Literal["1min", "3min"] = "1min", - calitate: Literal["Clară", "Mai mare ca impuls", "Slabă", "n/a"] = "n/a", - max_reached: Literal["SL_first", "TP0", "TP1", "TP2"] | None = None, - be_moved: bool | None = None, - screenshot_file: str | None = None, - note: str = "", -) -> dict: - """Build a M2DExtraction-compatible dict from minimal manual inputs. - - Derived fields: - - ora_utc from ora_ro (DST-aware) - - tp0/tp1/tp2 from entry/sl geometry - - risc_pct from |entry-sl|/entry - - max_reached/be_moved from outcome_path (overridable) - - screenshot_file generated from data+instrument+ora_ro if not provided - - The returned dict satisfies scripts.vision_schema.M2DExtraction. - """ - if entry == sl: - raise ValueError("entry == sl — zero risk distance") - - risk_abs = abs(entry - sl) - risc_pct = round(100 * risk_abs / entry, 4) - - if directie == "Buy": - if sl >= entry: - raise ValueError(f"Buy: sl ({sl}) must be < entry ({entry})") - tp0 = round(entry + 0.4 * risk_abs, 4) - tp1 = round(entry + 0.6 * risk_abs, 4) - tp2 = round(entry + risk_abs, 4) - else: # Sell - if sl <= entry: - raise ValueError(f"Sell: sl ({sl}) must be > entry ({entry})") - tp0 = round(entry - 0.4 * risk_abs, 4) - tp1 = round(entry - 0.6 * risk_abs, 4) - tp2 = round(entry - risk_abs, 4) - - ora_utc = ro_to_utc(data, ora_ro) - - if max_reached is None: - max_reached = OUTCOME_TO_MAX_REACHED[outcome_path] - if be_moved is None: - be_moved = OUTCOME_TO_BE_MOVED[outcome_path] - - if screenshot_file is None: - ora_compact = ora_ro.replace(":", "") - screenshot_file = f"{data}-{instrument.lower()}-{ora_compact}.png" - - return { - "screenshot_file": screenshot_file, - "data": data, - "ora_utc": ora_utc, - "instrument": instrument, - "directie": directie, - "tf_mare": tf_mare, - "tf_mic": tf_mic, - "calitate": calitate, - "entry": round(float(entry), 4), - "sl": round(float(sl), 4), - "tp0": tp0, - "tp1": tp1, - "tp2": tp2, - "risc_pct": risc_pct, - "outcome_path": outcome_path, - "max_reached": max_reached, - "be_moved": be_moved, - "confidence": "high", - "ambiguities": [], - "note": note, - } diff --git a/scripts/pl_calc.py b/scripts/pl_calc.py deleted file mode 100644 index 56676fe..0000000 --- a/scripts/pl_calc.py +++ /dev/null @@ -1,76 +0,0 @@ -"""P/L overlays for M2D backtesting. - -Two overlays computed from the same trade outcome: - -- ``pl_marius``: real overlay used by the trader. 50% closed at TP0 (+0.2 R), - BE move on the remaining half, then close 50% of that at ~TP1 (+0.3 R total - contribution) or at SL/BE depending on outcome. TP1 is treated as the final - exit even when the chart subsequently reaches TP2. - -- ``pl_theoretical``: reference 1/3-1/3-1/3 overlay that holds to TP2. Used - as an opportunity-cost benchmark vs. ``pl_marius``. - -Returns are expressed in multiples of R (risk per trade). ``None`` from -``pl_marius`` denotes a still-pending trade. -""" - -from __future__ import annotations - -__all__ = [ - "PL_MARIUS_TABLE", - "PL_THEORETICAL_TABLE", - "pl_marius", - "pl_theoretical", -] - - -PL_MARIUS_TABLE: dict[tuple[str, bool], float | None] = { - ("SL", True): -1.0, - ("SL", False): -1.0, - ("TP0->SL", True): 0.20, - ("TP0->SL", False): -0.30, - ("TP0->TP1", True): 0.50, - ("TP0->TP1", False): 0.50, - ("TP0->TP2", True): 0.50, - ("TP0->TP2", False): 0.50, - ("TP0->pending", True): None, - ("TP0->pending", False): None, - ("pending", True): None, - ("pending", False): None, -} - - -PL_THEORETICAL_TABLE: dict[str, float] = { - "SL_first": -1.0, - "TP0": 0.133, - "TP1": 0.333, - "TP2": 0.667, -} - - -_VALID_OUTCOME_PATHS: frozenset[str] = frozenset( - {"SL", "TP0->SL", "TP0->TP1", "TP0->TP2", "TP0->pending", "pending"} -) - - -def _normalize_outcome_path(outcome_path: str) -> str: - return outcome_path.replace("→", "->").replace("→", "->") - - -def pl_marius(outcome_path: str, be_moved: bool) -> float | None: - """Return the P/L (in R) for the real Marius overlay. - - Accepts both ASCII arrow ``"TP0->TP1"`` and unicode arrow ``"TP0→TP1"``. - Returns ``None`` for pending outcomes. - """ - normalized = _normalize_outcome_path(outcome_path) - if normalized not in _VALID_OUTCOME_PATHS: - raise ValueError(f"invalid outcome_path: {outcome_path!r}") - return PL_MARIUS_TABLE[(normalized, be_moved)] - - -def pl_theoretical(max_reached: str) -> float: - """Return the P/L (in R) for the theoretical 1/3-1/3-1/3 hold-to-TP2 overlay.""" - if max_reached not in PL_THEORETICAL_TABLE: - raise ValueError(f"invalid max_reached: {max_reached!r}") - return PL_THEORETICAL_TABLE[max_reached] diff --git a/scripts/regenerate_md.py b/scripts/regenerate_md.py deleted file mode 100644 index c799b5d..0000000 --- a/scripts/regenerate_md.py +++ /dev/null @@ -1,240 +0,0 @@ -"""Regenerate ``data/jurnal.md`` from ``data/jurnal.csv``. - -CSV is the source of truth (29 columns, schema owned by ``scripts.append_row``). -MD is a human-readable mirror with a curated 18-column table. - -CLI: ``python scripts/regenerate_md.py [csv_path] [md_path]`` -""" - -from __future__ import annotations - -import csv -import os -import sys -import tempfile -from datetime import datetime, timezone -from pathlib import Path -from typing import Sequence - -from scripts.append_row import csv_columns - -__all__ = ["MD_COLUMNS", "regenerate_md", "main"] - - -MD_COLUMNS: tuple[str, ...] = ( - "#", - "Data", - "Zi", - "Ora RO", - "Set", - "Instrument", - "Direcție", - "Calitate", - "Entry", - "SL", - "TP0", - "TP1", - "TP2", - "outcome_path", - "P/L (Marius)", - "P/L (theoretic)", - "Source", - "Note", -) - - -_CSV_FIELDS_USED: tuple[str, ...] = ( - "id", - "data", - "zi", - "ora_ro", - "set", - "instrument", - "directie", - "calitate", - "entry", - "sl", - "tp0", - "tp1", - "tp2", - "outcome_path", - "pl_marius", - "pl_theoretical", - "source", - "note", -) - - -_DIRECTIE_DISPLAY = {"long": "Buy", "short": "Sell", "buy": "Buy", "sell": "Sell"} - - -def _fmt_pl(value: str) -> str: - if value is None or value == "": - return "pending" - try: - return f"{float(value):+.2f}" - except ValueError: - return value - - -def _fmt_directie(value: str) -> str: - if not value: - return "" - return _DIRECTIE_DISPLAY.get(value.strip().lower(), value) - - -def _escape_cell(value: str) -> str: - return (value or "").replace("|", "\\|").replace("\n", " ").strip() - - -def _placeholder_md() -> str: - return ( - "# Jurnal M2D (auto-generated)\n" - "\n" - "*Niciun trade încă. Adaugă unul prin `/m2d-log` sau `/backtest`.*\n" - ) - - -def _atomic_write_text(path: Path, content: str) -> None: - path.parent.mkdir(parents=True, exist_ok=True) - fd, tmp_name = tempfile.mkstemp( - prefix=path.name + ".", suffix=".tmp", dir=str(path.parent) - ) - try: - with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as fh: - fh.write(content) - os.replace(tmp_name, path) - except Exception: - try: - os.unlink(tmp_name) - except OSError: - pass - raise - - -def _row_to_cells(row: dict[str, str], display_index: int) -> tuple[str, ...]: - g = row.get - return ( - str(display_index), - g("data", "") or "", - g("zi", "") or "", - g("ora_ro", "") or "", - g("set", "") or "", - g("instrument", "") or "", - _fmt_directie(g("directie", "") or ""), - g("calitate", "") or "", - g("entry", "") or "", - g("sl", "") or "", - g("tp0", "") or "", - g("tp1", "") or "", - g("tp2", "") or "", - g("outcome_path", "") or "", - _fmt_pl(g("pl_marius", "") or ""), - _fmt_pl(g("pl_theoretical", "") or ""), - g("source", "") or "", - g("note", "") or "", - ) - - -def _render_table(rows: Sequence[dict[str, str]]) -> str: - header_line = "| " + " | ".join(MD_COLUMNS) + " |" - sep_line = "|" + "|".join(["---"] * len(MD_COLUMNS)) + "|" - data_lines = [] - for i, row in enumerate(rows, start=1): - cells = _row_to_cells(row, i) - data_lines.append( - "| " + " | ".join(_escape_cell(c) for c in cells) + " |" - ) - return "\n".join([header_line, sep_line, *data_lines]) - - -def _render_md(rows: Sequence[dict[str, str]]) -> str: - if not rows: - return _placeholder_md() - now_iso = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") - table = _render_table(rows) - return ( - "# Jurnal M2D (auto-generated from data/jurnal.csv)\n" - "\n" - f"Generated: {now_iso}\n" - f"Rows: {len(rows)}\n" - "\n" - f"{table}\n" - "\n" - "*Vezi `data/jurnal.csv` pentru toate cele 29 coloane " - "(id, ora_utc, tf_*, risc_pct, be_moved, max_reached, versions, extracted_at).*\n" - ) - - -def _id_sort_key(raw: str) -> tuple[int, int | str]: - try: - return (0, int(raw)) - except (ValueError, TypeError): - return (1, raw or "") - - -def _load_rows(csv_path: Path) -> list[dict[str, str]]: - """Read CSV, returning rows sorted by id. - - Schema drift handling: - - Extra header columns → warning to stderr, dropped. - - Missing required header columns → warning to stderr per affected row (row skipped). - """ - if not csv_path.exists() or csv_path.stat().st_size == 0: - return [] - - expected = set(csv_columns()) - required = set(_CSV_FIELDS_USED) - - with csv_path.open("r", encoding="utf-8", newline="") as fh: - reader = csv.DictReader(fh) - header = reader.fieldnames or [] - header_set = set(header) - - extras = [c for c in header if c not in expected] - if extras: - print( - f"regenerate_md: warning: unknown CSV columns ignored: {extras}", - file=sys.stderr, - ) - - missing_required = required - header_set - rows: list[dict[str, str]] = [] - for raw in reader: - if missing_required: - print( - f"regenerate_md: warning: row skipped (missing required " - f"columns: {sorted(missing_required)})", - file=sys.stderr, - ) - continue - rows.append({k: (raw.get(k) or "") for k in required}) - - rows.sort(key=lambda r: _id_sort_key(r.get("id", ""))) - return rows - - -def regenerate_md( - csv_path: Path | str = "data/jurnal.csv", - md_path: Path | str = "data/jurnal.md", -) -> int: - """Read CSV → write MD atomically. Returns count of trade rows written.""" - csv_p = Path(csv_path) - md_p = Path(md_path) - rows = _load_rows(csv_p) - content = _render_md(rows) - _atomic_write_text(md_p, content) - return len(rows) - - -def main() -> int: - args = sys.argv[1:] - csv_arg = args[0] if len(args) >= 1 else "data/jurnal.csv" - md_arg = args[1] if len(args) >= 2 else "data/jurnal.md" - n = regenerate_md(csv_arg, md_arg) - print(f"regenerate_md: wrote {md_arg} with {n} row(s)") - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/scripts/stats.py b/scripts/stats.py deleted file mode 100644 index 24e7a6c..0000000 --- a/scripts/stats.py +++ /dev/null @@ -1,551 +0,0 @@ -"""Backtest statistics for ``data/jurnal.csv``. - -Public API: - - ``compute_stats(csv_path, overlay) -> dict`` - - ``render_stats(stats, overlay) -> str`` - - ``compute_calibration(csv_path) -> dict`` - - ``render_calibration(cal) -> str`` - - ``main()`` — CLI entry point. - -A "win" is a closed trade with ``pl_overlay > 0`` (where ``pl_overlay`` is -either ``pl_marius`` or ``pl_theoretical``). Pending trades — ``pl_marius`` -blank, i.e. ``outcome_path in {pending, TP0->pending}`` — are excluded from -both WR and expectancy: there is no realised outcome yet. - -The ``calitate`` field is a known-biased descriptor: it is classified -post-outcome (see ``STOPPING_RULE.md`` §3). The per-``calitate`` split is -reported with an explicit *descriptor only — biased post-outcome* caveat. -""" - -from __future__ import annotations - -import argparse -import csv -import math -import sys -from pathlib import Path -from typing import Any, Iterable - -import numpy as np - -from scripts.append_row import CSV_COLUMNS - -__all__ = [ - "BACKTEST_SOURCES", - "CALIBRATION_SOURCES", - "CORE_CALIBRATION_FIELDS", - "NUMERIC_CALIBRATION_FIELDS", - "STOPPING_RULE_N", - "wilson_ci", - "bootstrap_expectancy_ci", - "compute_stats", - "render_stats", - "compute_calibration", - "render_calibration", - "main", -] - - -# --------------------------------------------------------------------------- -# Constants -# --------------------------------------------------------------------------- - - -BACKTEST_SOURCES: frozenset[str] = frozenset({"vision", "manual"}) -CALIBRATION_SOURCES: frozenset[str] = frozenset( - {"manual_calibration", "vision_calibration"} -) - - -# Calibration P4 gate (STOPPING_RULE.md §P4) — explicitly reported per field. -CORE_CALIBRATION_FIELDS: tuple[str, ...] = ( - "entry", - "sl", - "tp0", - "tp1", - "tp2", - "outcome_path", - "max_reached", - "directie", - "instrument", -) - - -NUMERIC_CALIBRATION_FIELDS: frozenset[str] = frozenset( - {"entry", "sl", "tp0", "tp1", "tp2"} -) - - -# STOPPING_RULE.md §"GO LIVE" gate: N >= 40 per Set. -STOPPING_RULE_N: int = 40 - - -# --------------------------------------------------------------------------- -# Loading -# --------------------------------------------------------------------------- - - -def _parse_optional_float(value: str) -> float | None: - s = (value or "").strip() - if s == "": - return None - try: - return float(s) - except ValueError: - return None - - -def _load_rows(csv_path: Path | str) -> list[dict[str, str]]: - p = Path(csv_path) - if not p.exists() or p.stat().st_size == 0: - return [] - with p.open("r", encoding="utf-8", newline="") as fh: - return list(csv.DictReader(fh)) - - -# --------------------------------------------------------------------------- -# CI primitives -# --------------------------------------------------------------------------- - - -def wilson_ci(wins: int, n: int, z: float = 1.96) -> tuple[float, float]: - """Wilson score interval for a binomial proportion. - - Returns ``(lo, hi)`` clamped to ``[0.0, 1.0]``. For ``n == 0`` returns - ``(0.0, 0.0)``. ``z = 1.96`` ≈ 95% confidence. - """ - if n <= 0: - return (0.0, 0.0) - if wins < 0 or wins > n: - raise ValueError(f"wins={wins} out of range for n={n}") - p = wins / n - denom = 1.0 + (z * z) / n - center = (p + (z * z) / (2.0 * n)) / denom - spread = z * math.sqrt(p * (1.0 - p) / n + (z * z) / (4.0 * n * n)) / denom - return (max(0.0, center - spread), min(1.0, center + spread)) - - -def bootstrap_expectancy_ci( - values: list[float] | np.ndarray, - n_resamples: int = 5000, - seed: int = 42, -) -> tuple[float, float]: - """Percentile-method bootstrap 95% CI for the mean of ``values``. - - Deterministic for a given ``seed``. Empty input → ``(0.0, 0.0)``. - Single value → ``(value, value)`` (no variance to resample). - """ - arr = np.asarray(list(values), dtype=float) - if arr.size == 0: - return (0.0, 0.0) - if arr.size == 1: - v = float(arr[0]) - return (v, v) - rng = np.random.default_rng(seed) - boots = np.empty(n_resamples, dtype=float) - n = arr.size - for i in range(n_resamples): - idx = rng.integers(0, n, size=n) - boots[i] = float(arr[idx].mean()) - lo = float(np.percentile(boots, 2.5)) - hi = float(np.percentile(boots, 97.5)) - return (lo, hi) - - -# --------------------------------------------------------------------------- -# compute_stats -# --------------------------------------------------------------------------- - - -def _group_stats( - overlay_values: list[float | None], - *, - include_ci: bool, - bootstrap_seed: int, -) -> dict[str, Any]: - closed = [v for v in overlay_values if v is not None] - n = len(closed) - wins = sum(1 for v in closed if v > 0) - wr = (wins / n) if n else 0.0 - out: dict[str, Any] = { - "n": n, - "wr": wr, - "expectancy": (sum(closed) / n) if n else 0.0, - } - if include_ci: - out["wr_ci_95"] = wilson_ci(wins, n) - out["expectancy_ci_95"] = bootstrap_expectancy_ci( - closed, seed=bootstrap_seed - ) - return out - - -def _overlay_value(row: dict[str, str], overlay: str) -> float | None: - raw = row.get(overlay, "") - return _parse_optional_float(raw) - - -def compute_stats( - csv_path: Path | str = "data/jurnal.csv", - overlay: str = "pl_marius", -) -> dict[str, Any]: - """Compute aggregate WR + expectancy stats over the backtest rows. - - Calibration rows (``manual_calibration`` / ``vision_calibration``) are - excluded; use :func:`compute_calibration` for the P4 mismatch report. - - ``overlay`` selects the P/L column: ``"pl_marius"`` (default — the real - overlay Marius trades) or ``"pl_theoretical"`` (1/3-1/3-1/3 hold-to-TP2). - """ - if overlay not in {"pl_marius", "pl_theoretical"}: - raise ValueError(f"unknown overlay {overlay!r}") - - rows = [r for r in _load_rows(csv_path) if r.get("source", "") in BACKTEST_SOURCES] - - if not rows: - return { - "n_total": 0, - "n_pending": 0, - "n_closed": 0, - "wr": 0.0, - "wr_ci_95": (0.0, 0.0), - "expectancy": 0.0, - "expectancy_ci_95": (0.0, 0.0), - "per_set": {}, - "per_calitate": {}, - "per_directie": {}, - } - - # Pending status is overlay-independent: a trade is pending iff - # pl_marius is blank (outcome_path in {pending, TP0->pending}). - # pl_theoretical is concrete even for pending rows, so it would otherwise - # let pending trades sneak into the closed-trades stats — we mask those - # out explicitly here. - pending_mask = [_parse_optional_float(r.get("pl_marius", "")) is None for r in rows] - overlay_vals: list[float | None] = [] - for r, is_pending in zip(rows, pending_mask): - overlay_vals.append(None if is_pending else _overlay_value(r, overlay)) - n_total = len(rows) - n_pending = sum(1 for p in pending_mask if p) - n_closed = n_total - n_pending - - overall = _group_stats( - overlay_vals, include_ci=True, bootstrap_seed=42 - ) - - def _split(field: str, include_ci: bool) -> dict[str, dict[str, Any]]: - groups: dict[str, list[float | None]] = {} - for r, v in zip(rows, overlay_vals): - key = r.get(field, "") or "(blank)" - groups.setdefault(key, []).append(v) - out: dict[str, dict[str, Any]] = {} - for k in sorted(groups): - sub_seed = 42 + (abs(hash(("split", field, k))) % 1_000_000) - out[k] = _group_stats( - groups[k], include_ci=include_ci, bootstrap_seed=sub_seed - ) - return out - - return { - "n_total": n_total, - "n_pending": n_pending, - "n_closed": n_closed, - "wr": overall["wr"], - "wr_ci_95": overall["wr_ci_95"], - "expectancy": overall["expectancy"], - "expectancy_ci_95": overall["expectancy_ci_95"], - "per_set": _split("set", include_ci=True), - "per_calitate": _split("calitate", include_ci=True), - # per_directie skips CI per spec (no wr_ci_95 / expectancy_ci_95 keys). - "per_directie": { - k: {"n": v["n"], "wr": v["wr"], "expectancy": v["expectancy"]} - for k, v in _split("directie", include_ci=False).items() - }, - } - - -# --------------------------------------------------------------------------- -# render_stats -# --------------------------------------------------------------------------- - - -def _fmt_pct(p: float) -> str: - return f"{100.0 * p:5.1f}%" - - -def _fmt_r(x: float) -> str: - return f"{x:+.2f} R" - - -def _set_sort_key(name: str) -> tuple[int, str]: - order = ["A1", "A2", "A3", "B", "C", "D", "Other"] - return (order.index(name), name) if name in order else (len(order), name) - - -def render_stats(stats: dict[str, Any], overlay: str) -> str: - lines: list[str] = [] - lines.append(f"=== Stats jurnal.csv (overlay: {overlay}) ===") - lines.append( - f"Trade-uri totale: {stats['n_total']} | " - f"închise: {stats['n_closed']} | pending: {stats['n_pending']}" - ) - - if stats["n_total"] == 0: - lines.append("") - lines.append("(nu sunt trade-uri backtest în CSV)") - return "\n".join(lines) + "\n" - - lines.append("") - lo, hi = stats["wr_ci_95"] - e_lo, e_hi = stats["expectancy_ci_95"] - lines.append(f"GLOBAL (n={stats['n_closed']}):") - lines.append( - f" WR: {_fmt_pct(stats['wr'])} " - f"[95% CI: {_fmt_pct(lo)}, {_fmt_pct(hi)}]" - ) - lines.append( - f" Expectancy: {_fmt_r(stats['expectancy'])} " - f"[95% CI: {_fmt_r(e_lo)}, {_fmt_r(e_hi)}]" - ) - lines.append("") - - def _emit_split( - title: str, - data: dict[str, dict[str, Any]], - *, - sort_keys: list[str] | None = None, - include_ci: bool = True, - ) -> None: - lines.append(title) - keys = sort_keys if sort_keys is not None else sorted(data) - for k in keys: - if k not in data: - continue - d = data[k] - if include_ci and "wr_ci_95" in d: - clo, chi = d["wr_ci_95"] - lines.append( - f" {k:<14} n={d['n']:>3} " - f"WR {_fmt_pct(d['wr'])} " - f"[{_fmt_pct(clo)}, {_fmt_pct(chi)}] " - f"E {_fmt_r(d['expectancy'])}" - ) - else: - lines.append( - f" {k:<14} n={d['n']:>3} " - f"WR {_fmt_pct(d['wr'])} " - f"E {_fmt_r(d['expectancy'])}" - ) - lines.append("") - - _emit_split( - "PER SET:", - stats["per_set"], - sort_keys=sorted(stats["per_set"], key=_set_sort_key), - ) - - lines.append( - "PER CALITATE (⚠️ DESCRIPTOR ONLY — biased post-outcome, NU folosi ca filtru):" - ) - cal_order = ["Clară", "Mai mare ca impuls", "Slabă", "n/a"] - keys = [k for k in cal_order if k in stats["per_calitate"]] + [ - k for k in sorted(stats["per_calitate"]) if k not in cal_order - ] - for k in keys: - d = stats["per_calitate"][k] - clo, chi = d["wr_ci_95"] - lines.append( - f" {k:<20} n={d['n']:>3} " - f"WR {_fmt_pct(d['wr'])} " - f"[{_fmt_pct(clo)}, {_fmt_pct(chi)}] " - f"E {_fmt_r(d['expectancy'])}" - ) - lines.append("") - - _emit_split("PER DIRECȚIE:", stats["per_directie"], include_ci=False) - - # STOPPING_RULE gate check — flag every Set that hasn't crossed N>=40. - lines.append(f"⚠️ STOPPING RULE check (vezi STOPPING_RULE.md, N>={STOPPING_RULE_N}):") - set_keys = sorted(stats["per_set"], key=_set_sort_key) - any_flagged = False - for k in set_keys: - n = stats["per_set"][k]["n"] - if n < STOPPING_RULE_N: - lines.append(f" {k}: N={n} < {STOPPING_RULE_N} → NEEDS MORE DATA") - any_flagged = True - if not any_flagged: - lines.append(f" toate Set-urile au N>={STOPPING_RULE_N} (eligibile pentru GO LIVE check).") - - return "\n".join(lines) + "\n" - - -# --------------------------------------------------------------------------- -# compute_calibration -# --------------------------------------------------------------------------- - - -def _calibration_match(field: str, m_val: str, v_val: str, tol: float = 0.01) -> bool: - if field in NUMERIC_CALIBRATION_FIELDS: - try: - return abs(float(m_val) - float(v_val)) <= tol - except ValueError: - return (m_val or "").strip() == (v_val or "").strip() - return (m_val or "").strip() == (v_val or "").strip() - - -def compute_calibration( - csv_path: Path | str = "data/jurnal.csv", -) -> dict[str, Any]: - """Pair calibration legs by ``screenshot_file`` and report per-field mismatch. - - Returns a dict ``{"n_pairs": int, "fields": {field: {match, mismatch, - match_rate, mismatch_examples}}}``. ``mismatch_examples`` holds up to 3 - strings ``": manual=X vs vision=Y"`` per field. - - Numeric fields (``entry/sl/tp0/tp1/tp2``) use a tolerance of 0.01; - everything else is exact-string equality after strip. - """ - rows = _load_rows(csv_path) - manual: dict[str, dict[str, str]] = {} - vision: dict[str, dict[str, str]] = {} - for r in rows: - src = r.get("source", "") - if src == "manual_calibration": - manual[r.get("screenshot_file", "")] = r - elif src == "vision_calibration": - vision[r.get("screenshot_file", "")] = r - - paired_files = sorted(set(manual) & set(vision)) - fields_report: dict[str, dict[str, Any]] = { - f: { - "match": 0, - "mismatch": 0, - "match_rate": 0.0, - "mismatch_examples": [], - } - for f in CORE_CALIBRATION_FIELDS - } - - for f in paired_files: - m = manual[f] - v = vision[f] - for fld in CORE_CALIBRATION_FIELDS: - mv = m.get(fld, "") - vv = v.get(fld, "") - if _calibration_match(fld, mv, vv): - fields_report[fld]["match"] += 1 - else: - fields_report[fld]["mismatch"] += 1 - examples = fields_report[fld]["mismatch_examples"] - if len(examples) < 3: - examples.append(f"{f}: manual={mv!r} vs vision={vv!r}") - - for fld, data in fields_report.items(): - total = data["match"] + data["mismatch"] - data["match_rate"] = (data["match"] / total) if total else 0.0 - - return {"n_pairs": len(paired_files), "fields": fields_report} - - -def render_calibration(cal: dict[str, Any]) -> str: - lines: list[str] = [] - lines.append("=== Calibration P4 gate (vezi STOPPING_RULE.md §P4) ===") - lines.append(f"Perechi calibration: {cal['n_pairs']}") - if cal["n_pairs"] == 0: - lines.append("(nu există perechi manual_calibration ∩ vision_calibration)") - return "\n".join(lines) + "\n" - - lines.append("") - lines.append(f"{'field':<14} match mismatch rate") - total_mismatches = 0 - total_comparisons = 0 - for fld in CORE_CALIBRATION_FIELDS: - d = cal["fields"][fld] - n = d["match"] + d["mismatch"] - total_mismatches += d["mismatch"] - total_comparisons += n - lines.append( - f"{fld:<14} {d['match']:>5} {d['mismatch']:>8} " - f"{_fmt_pct(d['match_rate'])}" - ) - - lines.append("") - overall_match_rate = ( - (total_comparisons - total_mismatches) / total_comparisons - if total_comparisons - else 0.0 - ) - overall_mismatch_rate = 1.0 - overall_match_rate - verdict = "PASS" if overall_mismatch_rate <= 0.10 else "FAIL" - lines.append( - f"Overall mismatch rate: {_fmt_pct(overall_mismatch_rate)} " - f"({total_mismatches}/{total_comparisons}) → P4 gate: {verdict}" - ) - - has_examples = any( - cal["fields"][f]["mismatch_examples"] for f in CORE_CALIBRATION_FIELDS - ) - if has_examples: - lines.append("") - lines.append("Mismatch examples (max 3 per field):") - for fld in CORE_CALIBRATION_FIELDS: - ex = cal["fields"][fld]["mismatch_examples"] - if not ex: - continue - lines.append(f" [{fld}]") - for e in ex: - lines.append(f" - {e}") - - return "\n".join(lines) + "\n" - - -# --------------------------------------------------------------------------- -# CLI -# --------------------------------------------------------------------------- - - -def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser( - prog="stats", - description="Backtest statistics for data/jurnal.csv", - ) - parser.add_argument( - "--csv", - type=Path, - default=Path("data/jurnal.csv"), - help="Path to the jurnal CSV (default: data/jurnal.csv).", - ) - parser.add_argument( - "--overlay", - choices=("pl_marius", "pl_theoretical"), - default="pl_marius", - help="Which P/L overlay to use (default: pl_marius).", - ) - parser.add_argument( - "--calibration", - action="store_true", - help="Show P4 calibration mismatch report instead of backtest stats.", - ) - args = parser.parse_args(argv) - - try: - sys.stdout.reconfigure(encoding="utf-8") # type: ignore[attr-defined] - except (AttributeError, OSError): - pass - - if args.calibration: - cal = compute_calibration(args.csv) - sys.stdout.write(render_calibration(cal)) - else: - stats = compute_stats(args.csv, overlay=args.overlay) - sys.stdout.write(render_stats(stats, args.overlay)) - return 0 - - -if __name__ == "__main__": - raise SystemExit(main()) - - -# Ensure the canonical CSV schema is importable from one place — fail fast if -# someone removes append_row.CSV_COLUMNS that this module depends on. -assert CSV_COLUMNS is not None diff --git a/scripts/vision_schema.py b/scripts/vision_schema.py deleted file mode 100644 index 0f45032..0000000 --- a/scripts/vision_schema.py +++ /dev/null @@ -1,125 +0,0 @@ -"""Pydantic schema for the M2D vision-extraction JSON returned by the vision subagent.""" -from __future__ import annotations - -import re -from datetime import date as date_type, datetime, timezone -from typing import Literal - -from pydantic import BaseModel, ConfigDict, Field, model_validator - - -_DATA_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2}$") -_ORA_PATTERN = re.compile(r"^\d{2}:\d{2}$") - - -class M2DExtraction(BaseModel): - model_config = ConfigDict(extra="forbid") - - screenshot_file: str - data: str - ora_utc: str - instrument: Literal["DIA", "US30", "other"] - directie: Literal["Buy", "Sell"] - tf_mare: Literal["5min", "15min"] - tf_mic: Literal["1min", "3min"] - calitate: Literal["Clară", "Mai mare ca impuls", "Slabă", "n/a"] - entry: float - sl: float - tp0: float - tp1: float - tp2: float - risc_pct: float - outcome_path: Literal[ - "SL", "TP0→SL", "TP0→TP1", "TP0→TP2", "TP0→pending", "pending" - ] - max_reached: Literal["SL_first", "TP0", "TP1", "TP2"] - be_moved: bool - confidence: Literal["high", "medium", "low"] - ambiguities: list[str] = Field(default_factory=list) - note: str = "" - - @model_validator(mode="after") - def _validate_data_format(self) -> "M2DExtraction": - if not _DATA_PATTERN.match(self.data): - raise ValueError( - f"data must match YYYY-MM-DD, got {self.data!r}" - ) - try: - parsed = date_type.fromisoformat(self.data) - except ValueError as exc: - raise ValueError(f"data is not a valid ISO date: {self.data!r}") from exc - today = datetime.now(timezone.utc).date() - if parsed > today: - raise ValueError( - f"data {self.data!r} is in the future (today UTC: {today.isoformat()})" - ) - return self - - @model_validator(mode="after") - def _validate_ora_utc_format(self) -> "M2DExtraction": - if not _ORA_PATTERN.match(self.ora_utc): - raise ValueError( - f"ora_utc must match HH:MM, got {self.ora_utc!r}" - ) - try: - datetime.strptime(self.ora_utc, "%H:%M") - except ValueError as exc: - raise ValueError( - f"ora_utc is not a valid HH:MM time: {self.ora_utc!r}" - ) from exc - return self - - @model_validator(mode="after") - def _validate_entry_ne_sl(self) -> "M2DExtraction": - if self.entry == self.sl: - raise ValueError("entry must not equal sl (zero risk distance)") - return self - - @model_validator(mode="after") - def _validate_tp_ordering(self) -> "M2DExtraction": - if self.directie == "Buy": - if not (self.sl < self.entry < self.tp0 < self.tp1 < self.tp2): - raise ValueError( - "for Buy, required: sl < entry < tp0 < tp1 < tp2 " - f"(got sl={self.sl}, entry={self.entry}, tp0={self.tp0}, " - f"tp1={self.tp1}, tp2={self.tp2})" - ) - else: - if not (self.sl > self.entry > self.tp0 > self.tp1 > self.tp2): - raise ValueError( - "for Sell, required: sl > entry > tp0 > tp1 > tp2 " - f"(got sl={self.sl}, entry={self.entry}, tp0={self.tp0}, " - f"tp1={self.tp1}, tp2={self.tp2})" - ) - return self - - @model_validator(mode="after") - def _validate_outcome_max_consistency(self) -> "M2DExtraction": - op = self.outcome_path - mr = self.max_reached - if op == "SL": - if mr != "SL_first": - raise ValueError( - f"outcome_path='SL' requires max_reached='SL_first', got {mr!r}" - ) - elif op.startswith("TP0"): - if mr not in {"TP0", "TP1", "TP2"}: - raise ValueError( - f"outcome_path={op!r} requires max_reached in " - f"{{TP0, TP1, TP2}}, got {mr!r}" - ) - # op == "pending" → any max_reached accepted - return self - - -def parse_extraction(json_str: str) -> M2DExtraction: - """Parse a JSON string into an M2DExtraction. - - Raises pydantic.ValidationError on invalid input. - """ - return M2DExtraction.model_validate_json(json_str) - - -def parse_extraction_dict(d: dict) -> M2DExtraction: - """Validate a dict against the M2DExtraction schema.""" - return M2DExtraction.model_validate(d) diff --git a/tests/test_append_row.py b/tests/test_append_row.py deleted file mode 100644 index 7139a30..0000000 --- a/tests/test_append_row.py +++ /dev/null @@ -1,287 +0,0 @@ -"""Tests for scripts/append_row.py — append_extraction pipeline.""" - -from __future__ import annotations - -import csv -import json -import re -import sys -from datetime import datetime -from pathlib import Path - -import pytest -import yaml - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.append_row import ( # noqa: E402 - CSV_COLUMNS, - VALID_SOURCES, - ZI_RO_MAP, - append_extraction, - csv_columns, -) - - -REPO_ROOT = Path(__file__).resolve().parent.parent -CALENDAR_PATH = REPO_ROOT / "calendar_evenimente.yaml" -META_PATH = REPO_ROOT / "data" / "_meta.yaml" - - -# --------------------------------------------------------------------------- -# helpers / fixtures -# --------------------------------------------------------------------------- - - -def _buy_payload(**overrides) -> dict: - # 2026-05-13 14:23 UTC == 17:23 RO (EEST, Wed) → set A2, zi=Mi. - base = { - "screenshot_file": "dia-2026-05-13-1.png", - "data": "2026-05-13", - "ora_utc": "14:23", - "instrument": "DIA", - "directie": "Buy", - "tf_mare": "5min", - "tf_mic": "1min", - "calitate": "Clară", - "entry": 400.0, - "sl": 399.0, - "tp0": 400.5, - "tp1": 401.0, - "tp2": 402.0, - "risc_pct": 0.25, - "outcome_path": "TP0→TP1", - "max_reached": "TP1", - "be_moved": True, - "confidence": "high", - "ambiguities": [], - "note": "", - } - base.update(overrides) - return base - - -def _write_payload(tmp_path: Path, name: str, **overrides) -> Path: - p = tmp_path / name - p.write_text(json.dumps(_buy_payload(**overrides)), encoding="utf-8") - return p - - -def _read_rows(csv_path: Path) -> list[dict[str, str]]: - with csv_path.open("r", encoding="utf-8", newline="") as fh: - return list(csv.DictReader(fh)) - - -@pytest.fixture -def csv_path(tmp_path: Path) -> Path: - return tmp_path / "jurnal.csv" - - -# --------------------------------------------------------------------------- -# schema / column layout -# --------------------------------------------------------------------------- - - -def test_csv_columns_canonical_29() -> None: - cols = csv_columns() - assert len(cols) == 29 - assert cols[0] == "id" - assert cols[-1] == "note" - assert cols == list(CSV_COLUMNS) - - -# --------------------------------------------------------------------------- -# core tests as specified in task #9 -# --------------------------------------------------------------------------- - - -def test_happy_path(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - result = append_extraction( - j, "vision", csv_path, META_PATH, CALENDAR_PATH - ) - assert result["status"] == "ok", result - assert result["reason"] == "" - assert result["id"] == 1 - - rows = _read_rows(csv_path) - assert len(rows) == 1 - r = rows[0] - assert r["id"] == "1" - assert r["screenshot_file"] == "dia-2026-05-13-1.png" - assert r["source"] == "vision" - assert r["data"] == "2026-05-13" - assert r["zi"] == "Mi" - assert r["ora_ro"] == "17:23" - assert r["ora_utc"] == "14:23" - assert r["set"] == "A2" - assert r["instrument"] == "DIA" - assert r["directie"] == "Buy" - assert r["be_moved"] == "True" - - -def test_pl_calc_overlay(tmp_path: Path, csv_path: Path) -> None: - """outcome_path=TP0->TP1, max_reached=TP1 → pl_marius=0.5, pl_theoretical=0.333.""" - j = _write_payload(tmp_path, "t.json") - result = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert result["status"] == "ok" - r = _read_rows(csv_path)[0] - assert float(r["pl_marius"]) == pytest.approx(0.50) - assert float(r["pl_theoretical"]) == pytest.approx(0.333) - - -def test_dedup_same_source(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - r1 = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - r2 = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert r1["status"] == "ok" - assert r2["status"] == "rejected" - assert "duplicate" in r2["reason"].lower() - assert r2["id"] is None - assert r2["row"] is None - assert len(_read_rows(csv_path)) == 1 - - -def test_dedup_different_source_ok(tmp_path: Path, csv_path: Path) -> None: - """Same screenshot_file + different source ⇒ both rows accepted.""" - j = _write_payload(tmp_path, "t.json") - r1 = append_extraction( - j, "manual_calibration", csv_path, META_PATH, CALENDAR_PATH - ) - r2 = append_extraction( - j, "vision_calibration", csv_path, META_PATH, CALENDAR_PATH - ) - assert r1["status"] == "ok" - assert r2["status"] == "ok" - rows = _read_rows(csv_path) - assert len(rows) == 2 - assert {r["source"] for r in rows} == {"manual_calibration", "vision_calibration"} - # Distinct sequential ids. - assert {r["id"] for r in rows} == {"1", "2"} - - -def test_invalid_pydantic_rejected(tmp_path: Path, csv_path: Path) -> None: - """entry == sl is rejected by pydantic; no CSV is written.""" - j = _write_payload(tmp_path, "bad.json", entry=399.0, sl=399.0) - result = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert result["status"] == "rejected" - assert "validation" in result["reason"].lower() - assert not csv_path.exists() - - -def test_missing_json_file(tmp_path: Path, csv_path: Path) -> None: - missing = tmp_path / "ghost.json" - result = append_extraction( - missing, "vision", csv_path, META_PATH, CALENDAR_PATH - ) - assert result["status"] == "rejected" - assert "not found" in result["reason"].lower() - assert not csv_path.exists() - - -def test_id_increments(tmp_path: Path, csv_path: Path) -> None: - paths = [ - _write_payload(tmp_path, "a.json", screenshot_file="a.png"), - _write_payload(tmp_path, "b.json", screenshot_file="b.png"), - _write_payload(tmp_path, "c.json", screenshot_file="c.png"), - ] - ids = [] - for p in paths: - r = append_extraction(p, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "ok" - ids.append(r["id"]) - assert ids == [1, 2, 3] - csv_ids = [int(r["id"]) for r in _read_rows(csv_path)] - assert csv_ids == [1, 2, 3] - - -def test_set_a2(tmp_path: Path, csv_path: Path) -> None: - """Wed 2026-05-13 14:30 UTC → 17:30 RO → A2 sweet spot.""" - j = _write_payload(tmp_path, "t.json", ora_utc="14:30") - r = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "ok" - row = _read_rows(csv_path)[0] - assert row["ora_ro"] == "17:30" - assert row["zi"] == "Mi" - assert row["set"] == "A2" - - -def test_set_c_fomc(tmp_path: Path, csv_path: Path) -> None: - """2026-04-29 18:35 UTC == 21:35 RO (FOMC Powell Press window) → Set C.""" - j = _write_payload( - tmp_path, - "t.json", - data="2026-04-29", - ora_utc="18:35", - screenshot_file="fomc-apr.png", - ) - r = append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "ok" - row = _read_rows(csv_path)[0] - assert row["ora_ro"] == "21:35" - assert row["set"] == "C" - - -def test_versions_stamped(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - row = _read_rows(csv_path)[0] - meta = yaml.safe_load(META_PATH.read_text(encoding="utf-8")) - assert row["indicator_version"] == str(meta["indicator_version"]) - assert row["pl_overlay_version"] == str(meta["pl_overlay_version"]) - assert row["csv_schema_version"] == str(meta["csv_schema_version"]) - - -def test_extracted_at_format(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - val = _read_rows(csv_path)[0]["extracted_at"] - # ISO 8601 UTC with trailing 'Z': YYYY-MM-DDTHH:MM:SSZ - assert re.match(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$", val), val - # Round-trip through datetime.fromisoformat after dropping the Z. - parsed = datetime.fromisoformat(val[:-1]) - assert parsed.year >= 2026 - - -# --------------------------------------------------------------------------- -# additional safety nets -# --------------------------------------------------------------------------- - - -def test_invalid_source_rejected(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - r = append_extraction(j, "auto_magic", csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "rejected" - assert "source" in r["reason"].lower() - assert not csv_path.exists() - - -def test_all_valid_sources_accepted(tmp_path: Path, csv_path: Path) -> None: - for i, src in enumerate(sorted(VALID_SOURCES)): - j = _write_payload(tmp_path, f"t{i}.json", screenshot_file=f"s{i}.png") - r = append_extraction(j, src, csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "ok", (src, r) - rows = _read_rows(csv_path) - assert {r["source"] for r in rows} == set(VALID_SOURCES) - - -def test_atomic_write_leaves_no_tmp(tmp_path: Path, csv_path: Path) -> None: - j = _write_payload(tmp_path, "t.json") - append_extraction(j, "vision", csv_path, META_PATH, CALENDAR_PATH) - leftovers = [p for p in csv_path.parent.iterdir() if p.name.endswith(".tmp")] - assert leftovers == [] - - -def test_zi_ro_map_covers_all_weekdays() -> None: - """Internal sanity: the Romanian-day map covers all 7 short weekday names.""" - assert set(ZI_RO_MAP.keys()) == {"Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"} - assert set(ZI_RO_MAP.values()) == {"Lu", "Ma", "Mi", "Jo", "Vi", "Sa", "Du"} - - -def test_malformed_json_rejected(tmp_path: Path, csv_path: Path) -> None: - bad = tmp_path / "broken.json" - bad.write_text("{not valid json", encoding="utf-8") - r = append_extraction(bad, "vision", csv_path, META_PATH, CALENDAR_PATH) - assert r["status"] == "rejected" - assert "validation" in r["reason"].lower() or "json" in r["reason"].lower() - assert not csv_path.exists() diff --git a/tests/test_calendar_yaml.py b/tests/test_calendar_yaml.py deleted file mode 100644 index 17e7abc..0000000 --- a/tests/test_calendar_yaml.py +++ /dev/null @@ -1,88 +0,0 @@ -"""Tests for the YAML loader and news-window logic in calendar_parse.""" - -from __future__ import annotations - -import sys -import textwrap -from datetime import date, time -from pathlib import Path - -import pytest - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.calendar_parse import ( # noqa: E402 - is_in_news_window, - load_calendar, -) - - -REPO_ROOT = Path(__file__).resolve().parent.parent -CALENDAR_PATH = REPO_ROOT / "calendar_evenimente.yaml" - - -def test_load_calendar() -> None: - events = load_calendar(CALENDAR_PATH) - assert isinstance(events, list) - assert len(events) > 0 - required = {"name", "cadence", "time_ro", "severity", "window_before_min", "window_after_min"} - for ev in events: - missing = required - set(ev.keys()) - assert not missing, f"event {ev.get('name')!r} missing fields: {missing}" - - -def test_load_calendar_bad_version(tmp_path: Path) -> None: - bad = tmp_path / "bad.yaml" - bad.write_text( - textwrap.dedent( - """ - schema_version: 99 - events: [] - """ - ).strip() - + "\n", - encoding="utf-8", - ) - with pytest.raises(ValueError): - load_calendar(bad) - - -def _scheduled(date_str: str, time_str: str, before: int, after: int, severity: str = "extrem") -> dict: - return { - "name": "Test", - "cadence": "scheduled", - "date": date_str, - "time_ro": time_str, - "severity": severity, - "window_before_min": before, - "window_after_min": after, - } - - -class TestWindowBoundaries: - def setup_method(self) -> None: - self.cal = [_scheduled("2026-05-06", "15:30", 15, 15)] - self.d = date(2026, 5, 6) - - def test_window_inside_boundary(self) -> None: - assert is_in_news_window(self.d, time(15, 14), self.cal) is False # 1 min before lower bound - assert is_in_news_window(self.d, time(15, 15), self.cal) is True # lower bound inclusive - assert is_in_news_window(self.d, time(15, 45), self.cal) is True # upper bound inclusive - - def test_window_outside(self) -> None: - assert is_in_news_window(self.d, time(15, 14), self.cal) is False - assert is_in_news_window(self.d, time(15, 46), self.cal) is False - - -def test_severity_filter_mediu_excluded() -> None: - # JOLTS-like event with severity 'mediu' at 17:00 — even smack on time, no Set C trigger. - cal = [_scheduled("2026-05-06", "17:00", 10, 10, severity="mediu")] - assert is_in_news_window(date(2026, 5, 6), time(17, 0), cal) is False - assert is_in_news_window(date(2026, 5, 6), time(17, 5), cal) is False - - -def test_fomc_powell_window() -> None: - """Real FOMC Powell Press Apr from calendar_evenimente.yaml (2026-04-29 21:30 RO, 0/45).""" - cal = load_calendar(CALENDAR_PATH) - assert is_in_news_window(date(2026, 4, 29), time(21, 35), cal) is True - assert is_in_news_window(date(2026, 4, 29), time(22, 16), cal) is False diff --git a/tests/test_manual_log.py b/tests/test_manual_log.py deleted file mode 100644 index 006752f..0000000 --- a/tests/test_manual_log.py +++ /dev/null @@ -1,175 +0,0 @@ -"""Tests for scripts.manual_log — derivation logic + vision_schema compatibility.""" - -from __future__ import annotations - -import pytest - -from scripts.manual_log import build_extraction, OUTCOME_TO_BE_MOVED, OUTCOME_TO_MAX_REACHED, ro_to_utc -from scripts.vision_schema import M2DExtraction - - -def test_buy_happy_path(): - d = build_extraction( - data="2026-05-13", - ora_ro="17:33", - directie="Buy", - entry=497.42, - sl=496.80, - outcome_path="TP0→TP1", - ) - # Satisfies pydantic - M2DExtraction.model_validate(d) - assert d["directie"] == "Buy" - assert d["entry"] == 497.42 - assert d["sl"] == 496.80 - # tp0 = entry + 0.4 * 0.62 = 497.42 + 0.248 = 497.668 - assert abs(d["tp0"] - 497.668) < 0.01 - assert abs(d["tp1"] - 497.792) < 0.01 - assert abs(d["tp2"] - 498.04) < 0.01 - assert d["max_reached"] == "TP1" - assert d["be_moved"] is True - - -def test_sell_happy_path(): - d = build_extraction( - data="2026-05-13", - ora_ro="17:33", - directie="Sell", - entry=492.47, - sl=492.77, - outcome_path="TP0→TP1", - ) - M2DExtraction.model_validate(d) - assert d["directie"] == "Sell" - # For Sell: tp0 = entry - 0.4 * 0.30 = 492.47 - 0.12 = 492.35 - assert abs(d["tp0"] - 492.35) < 0.01 - assert abs(d["tp1"] - 492.29) < 0.01 - assert abs(d["tp2"] - 492.17) < 0.01 - assert d["sl"] > d["entry"] > d["tp0"] > d["tp1"] > d["tp2"] - - -def test_ora_utc_dst_summer(): - # May = EEST = UTC+3 - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Sell", - entry=492.47, sl=492.77, outcome_path="pending", - ) - assert d["ora_utc"] == "14:33" - - -def test_ora_utc_dst_winter(): - # January = EET = UTC+2 - d = build_extraction( - data="2026-01-15", ora_ro="17:00", directie="Buy", - entry=400, sl=399, outcome_path="pending", - ) - assert d["ora_utc"] == "15:00" - - -def test_outcome_sl_max_reached_consistent(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=399, outcome_path="SL", - ) - M2DExtraction.model_validate(d) - assert d["max_reached"] == "SL_first" - assert d["be_moved"] is False - - -def test_outcome_tp0_pending_max_reached_tp0(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=399, outcome_path="TP0→pending", - ) - M2DExtraction.model_validate(d) - assert d["max_reached"] == "TP0" - assert d["be_moved"] is True - - -def test_outcome_tp0_sl_be_moved_true(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=399, outcome_path="TP0→SL", - ) - M2DExtraction.model_validate(d) - assert d["max_reached"] == "TP0" - assert d["be_moved"] is True # rule-enforced - - -def test_explicit_max_reached_override(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=399, outcome_path="pending", - max_reached="TP0", - ) - M2DExtraction.model_validate(d) - assert d["max_reached"] == "TP0" - - -def test_screenshot_file_generated(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Sell", - entry=492.47, sl=492.77, outcome_path="pending", - instrument="DIA", - ) - assert d["screenshot_file"] == "2026-05-13-dia-1733.png" - - -def test_screenshot_file_explicit(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Sell", - entry=492.47, sl=492.77, outcome_path="pending", - screenshot_file="custom.png", - ) - assert d["screenshot_file"] == "custom.png" - - -def test_risc_pct_computed(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Sell", - entry=500.00, sl=501.00, outcome_path="pending", - ) - assert abs(d["risc_pct"] - 0.20) < 0.001 # 1/500 = 0.002 = 0.20% - - -def test_entry_equals_sl_raises(): - with pytest.raises(ValueError, match="zero risk"): - build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=500, sl=500, outcome_path="pending", - ) - - -def test_buy_inverted_ordering_raises(): - with pytest.raises(ValueError, match="Buy"): - build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=401, outcome_path="pending", # sl > entry for Buy - ) - - -def test_sell_inverted_ordering_raises(): - with pytest.raises(ValueError, match="Sell"): - build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Sell", - entry=400, sl=399, outcome_path="pending", # sl < entry for Sell - ) - - -def test_optional_fields_defaults(): - d = build_extraction( - data="2026-05-13", ora_ro="17:33", directie="Buy", - entry=400, sl=399, outcome_path="pending", - ) - assert d["instrument"] == "DIA" - assert d["tf_mare"] == "5min" - assert d["tf_mic"] == "1min" - assert d["calitate"] == "n/a" - assert d["confidence"] == "high" - assert d["ambiguities"] == [] - assert d["note"] == "" - - -def test_ro_to_utc_helper(): - assert ro_to_utc("2026-05-13", "17:33") == "14:33" - assert ro_to_utc("2026-01-15", "10:00") == "08:00" diff --git a/tests/test_pl_calc.py b/tests/test_pl_calc.py deleted file mode 100644 index 4bcb117..0000000 --- a/tests/test_pl_calc.py +++ /dev/null @@ -1,89 +0,0 @@ -"""Tests for scripts/pl_calc.py.""" - -from __future__ import annotations - -import sys -from pathlib import Path - -import pytest - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.pl_calc import ( # noqa: E402 - PL_MARIUS_TABLE, - PL_THEORETICAL_TABLE, - pl_marius, - pl_theoretical, -) - - -class TestPlMarius: - def test_sl(self) -> None: - assert pl_marius("SL", be_moved=True) == -1.0 - assert pl_marius("SL", be_moved=False) == -1.0 - - def test_tp0_sl_be_moved(self) -> None: - assert pl_marius("TP0->SL", be_moved=True) == pytest.approx(0.20) - - def test_tp0_sl_no_be(self) -> None: - assert pl_marius("TP0->SL", be_moved=False) == pytest.approx(-0.30) - - def test_tp0_tp1(self) -> None: - assert pl_marius("TP0->TP1", be_moved=True) == pytest.approx(0.50) - assert pl_marius("TP0->TP1", be_moved=False) == pytest.approx(0.50) - - def test_tp0_tp2_closes_at_tp1(self) -> None: - assert pl_marius("TP0->TP2", be_moved=True) == pytest.approx(0.50) - assert pl_marius("TP0->TP2", be_moved=False) == pytest.approx(0.50) - - def test_tp0_pending_returns_none(self) -> None: - assert pl_marius("TP0->pending", be_moved=True) is None - assert pl_marius("TP0->pending", be_moved=False) is None - - def test_pending_returns_none(self) -> None: - assert pl_marius("pending", be_moved=True) is None - assert pl_marius("pending", be_moved=False) is None - - def test_unicode_arrow_accepted(self) -> None: - assert pl_marius("TP0→TP1", be_moved=True) == pytest.approx(0.50) - assert pl_marius("TP0→SL", be_moved=False) == pytest.approx(-0.30) - - def test_invalid_outcome_path(self) -> None: - with pytest.raises(ValueError): - pl_marius("nonsense", be_moved=True) - with pytest.raises(ValueError): - pl_marius("TP3", be_moved=False) - with pytest.raises(ValueError): - pl_marius("", be_moved=True) - - -class TestPlTheoretical: - def test_sl_first(self) -> None: - assert pl_theoretical("SL_first") == -1.0 - - def test_tp0(self) -> None: - assert pl_theoretical("TP0") == pytest.approx(0.133) - - def test_tp1(self) -> None: - assert pl_theoretical("TP1") == pytest.approx(0.333) - - def test_tp2(self) -> None: - assert pl_theoretical("TP2") == pytest.approx(0.667) - - def test_invalid_max_reached(self) -> None: - with pytest.raises(ValueError): - pl_theoretical("TP3") - with pytest.raises(ValueError): - pl_theoretical("sl_first") # case-sensitive - with pytest.raises(ValueError): - pl_theoretical("") - - -class TestTables: - def test_marius_table_exported(self) -> None: - assert ("SL", True) in PL_MARIUS_TABLE - assert PL_MARIUS_TABLE[("TP0->TP1", True)] == pytest.approx(0.50) - - def test_theoretical_table_exported(self) -> None: - assert PL_THEORETICAL_TABLE["TP2"] == pytest.approx(0.667) - assert PL_THEORETICAL_TABLE["SL_first"] == -1.0 diff --git a/tests/test_regenerate_md.py b/tests/test_regenerate_md.py deleted file mode 100644 index a9f0f6b..0000000 --- a/tests/test_regenerate_md.py +++ /dev/null @@ -1,208 +0,0 @@ -"""Tests for scripts/regenerate_md.py.""" - -from __future__ import annotations - -import csv -import sys -from pathlib import Path - -import pytest - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.append_row import csv_columns # noqa: E402 -from scripts.regenerate_md import MD_COLUMNS, regenerate_md # noqa: E402 - - -def _row(**overrides: str) -> dict[str, str]: - base = { - "id": "1", - "screenshot_file": "2026-05-13_dia_5min.png", - "source": "vision", - "data": "2026-05-13", - "zi": "Mi", - "ora_ro": "17:23", - "ora_utc": "14:23", - "instrument": "DIA", - "directie": "long", - "tf_mare": "5min", - "tf_mic": "1min", - "calitate": "Clară", - "entry": "497.42", - "sl": "496.80", - "tp0": "497.67", - "tp1": "497.79", - "tp2": "498.04", - "risc_pct": "0.50", - "outcome_path": "TP0→TP1", - "max_reached": "TP1", - "be_moved": "true", - "pl_marius": "0.5000", - "pl_theoretical": "0.3330", - "set": "A2", - "indicator_version": "1", - "pl_overlay_version": "1", - "csv_schema_version": "1", - "extracted_at": "2026-05-13T14:30:00Z", - "note": "", - } - base.update(overrides) - return base - - -def _write_csv( - path: Path, - rows: list[dict[str, str]], - extra_columns: list[str] | None = None, -) -> None: - fieldnames = csv_columns() - if extra_columns: - fieldnames = fieldnames + extra_columns - with path.open("w", encoding="utf-8", newline="") as fh: - writer = csv.DictWriter(fh, fieldnames=fieldnames) - writer.writeheader() - for r in rows: - writer.writerow({k: r.get(k, "") for k in fieldnames}) - - -def _data_lines(md_text: str) -> list[str]: - header_prefix = "| " + MD_COLUMNS[0] + " | " + MD_COLUMNS[1] - return [ - ln - for ln in md_text.splitlines() - if ln.startswith("|") - and not ln.startswith(header_prefix) - and not ln.startswith("|---") - ] - - -def test_empty_csv_placeholder(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, []) - - n = regenerate_md(csv_p, md_p) - - assert n == 0 - content = md_p.read_text(encoding="utf-8") - assert "# Jurnal M2D (auto-generated)" in content - assert "Niciun trade încă" in content - assert "| # |" not in content - - -def test_missing_csv_placeholder(tmp_path: Path) -> None: - csv_p = tmp_path / "does_not_exist.csv" - md_p = tmp_path / "jurnal.md" - - n = regenerate_md(csv_p, md_p) - - assert n == 0 - content = md_p.read_text(encoding="utf-8") - assert "Niciun trade încă" in content - assert md_p.exists() - - -def test_single_row_format(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, [_row()]) - - n = regenerate_md(csv_p, md_p) - - assert n == 1 - content = md_p.read_text(encoding="utf-8") - assert "# Jurnal M2D (auto-generated from data/jurnal.csv)" in content - assert "Rows: 1" in content - header_line = "| " + " | ".join(MD_COLUMNS) + " |" - assert header_line in content - rows = _data_lines(content) - assert len(rows) == 1 - cells = [c.strip() for c in rows[0].strip("|").split("|")] - assert cells[0] == "1" - assert cells[1] == "2026-05-13" - assert cells[2] == "Mi" - assert cells[3] == "17:23" - assert cells[4] == "A2" - assert cells[5] == "DIA" - assert cells[6] == "Buy" - assert cells[7] == "Clară" - assert cells[13] == "TP0→TP1" - assert cells[14] == "+0.50" - assert cells[15] == "+0.33" - assert cells[16] == "vision" - - -def test_three_rows(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - rows = [ - _row(id="3", data="2026-05-15", pl_marius="-1.0000"), - _row(id="1", data="2026-05-13"), - _row(id="2", data="2026-05-14", pl_marius="0.2000"), - ] - _write_csv(csv_p, rows) - - n = regenerate_md(csv_p, md_p) - - assert n == 3 - content = md_p.read_text(encoding="utf-8") - assert "Rows: 3" in content - data = _data_lines(content) - assert len(data) == 3 - assert "| 1 | 2026-05-13 |" in data[0] - assert "| 2 | 2026-05-14 |" in data[1] - assert "| 3 | 2026-05-15 |" in data[2] - - -def test_pending_pl_displayed(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, [_row(pl_marius="", pl_theoretical="")]) - - n = regenerate_md(csv_p, md_p) - - assert n == 1 - content = md_p.read_text(encoding="utf-8") - rows = _data_lines(content) - cells = [c.strip() for c in rows[0].strip("|").split("|")] - assert cells[14] == "pending" - assert cells[15] == "pending" - - -def test_unknown_column_graceful( - tmp_path: Path, capsys: pytest.CaptureFixture[str] -) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, [_row()], extra_columns=["extra_field"]) - - n = regenerate_md(csv_p, md_p) - - assert n == 1 - content = md_p.read_text(encoding="utf-8") - assert "Rows: 1" in content - captured = capsys.readouterr() - assert "unknown CSV columns ignored" in captured.err - assert "extra_field" in captured.err - - -def test_atomic_write_no_tmp_leftover(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, [_row()]) - - regenerate_md(csv_p, md_p) - - leftovers = list(tmp_path.glob("*.tmp")) - assert leftovers == [] - assert md_p.exists() - - -def test_rows_count_returned(tmp_path: Path) -> None: - csv_p = tmp_path / "jurnal.csv" - md_p = tmp_path / "jurnal.md" - _write_csv(csv_p, [_row(id=str(i)) for i in range(1, 6)]) - - n = regenerate_md(csv_p, md_p) - - assert n == 5 diff --git a/tests/test_set_calc.py b/tests/test_set_calc.py deleted file mode 100644 index 6e4a6d5..0000000 --- a/tests/test_set_calc.py +++ /dev/null @@ -1,88 +0,0 @@ -"""Tests for calc_set + utc_to_ro in calendar_parse.""" - -from __future__ import annotations - -import sys -from datetime import date, time -from pathlib import Path - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.calendar_parse import ( # noqa: E402 - calc_set, - load_calendar, - utc_to_ro, -) - - -REPO_ROOT = Path(__file__).resolve().parent.parent -CALENDAR_PATH = REPO_ROOT / "calendar_evenimente.yaml" - - -def _cal(): - return load_calendar(CALENDAR_PATH) - - -# Reference weekdays used below (verified via datetime): -# 2026-05-13 Wed 2026-05-12 Tue 2026-05-14 Thu -# 2026-05-11 Mon 2026-05-15 Fri -# 2026-04-29 Wed (FOMC Powell Press Apr — Set C trigger) - - -def test_a1_mid() -> None: - assert calc_set(date(2026, 5, 13), time(16, 50), "Wed", _cal()) == "A1" - - -def test_a1_boundary_low() -> None: - assert calc_set(date(2026, 5, 12), time(16, 35), "Tue", _cal()) == "A1" - - -def test_a1_boundary_high() -> None: - assert calc_set(date(2026, 5, 14), time(16, 59), "Thu", _cal()) == "A1" - - -def test_a2_sweet_spot() -> None: - assert calc_set(date(2026, 5, 13), time(17, 30), "Wed", _cal()) == "A2" - - -def test_a3() -> None: - assert calc_set(date(2026, 5, 12), time(18, 30), "Tue", _cal()) == "A3" - - -def test_b() -> None: - assert calc_set(date(2026, 5, 14), time(22, 15), "Thu", _cal()) == "B" - - -def test_c_fomc() -> None: - # 2026-04-29 is Wed; would otherwise hit a time band — but FOMC Powell Press window dominates. - assert calc_set(date(2026, 4, 29), time(21, 35), "Wed", _cal()) == "C" - - -def test_d_mon() -> None: - assert calc_set(date(2026, 5, 11), time(17, 0), "Mon", _cal()) == "D" - - -def test_d_fri() -> None: - assert calc_set(date(2026, 5, 15), time(17, 0), "Fri", _cal()) == "D" - - -def test_other() -> None: - # Tue 13:00 — not Mon/Fri, no news, before any A-band. - assert calc_set(date(2026, 5, 12), time(13, 0), "Tue", _cal()) == "Other" - - -def test_dst_boundary_oct_2026() -> None: - """DST ends on Sun 2026-10-25 at 04:00 RO (clocks go back to 03:00). - - Just before the shift, 00:30 UTC = 03:30 RO (EEST, UTC+3). The conversion must - pick the pre-shift offset and yield 03:30 — not 02:30 (which would be an - off-by-one-hour bug from naive +2h). - """ - d_ro, t_ro, dow = utc_to_ro("2026-10-25", "00:30") - assert d_ro == date(2026, 10, 25) - assert t_ro == time(3, 30) - assert dow == "Sun" - - # After the shift, 01:30 UTC also maps to 03:30 RO (EET, UTC+2) — sanity check. - d_ro2, t_ro2, _ = utc_to_ro("2026-10-25", "01:30") - assert (d_ro2, t_ro2) == (date(2026, 10, 25), time(3, 30)) diff --git a/tests/test_stats.py b/tests/test_stats.py deleted file mode 100644 index bd12a41..0000000 --- a/tests/test_stats.py +++ /dev/null @@ -1,447 +0,0 @@ -"""CSV-fixture tests for scripts.stats — compute_stats, render_stats, -compute_calibration, render_calibration, main().""" - -from __future__ import annotations - -import csv -import sys -from pathlib import Path - -import pytest - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.append_row import CSV_COLUMNS # noqa: E402 -from scripts.stats import ( # noqa: E402 - CORE_CALIBRATION_FIELDS, - compute_calibration, - compute_stats, - main, - render_calibration, - render_stats, -) - - -# --------------------------------------------------------------------------- -# Fixture row builder -# --------------------------------------------------------------------------- - - -def _base_row(**overrides) -> dict[str, str]: - base = { - "id": "0", - "screenshot_file": "", - "source": "vision", - "data": "2026-05-13", - "zi": "Mi", - "ora_ro": "17:30", - "ora_utc": "14:30", - "instrument": "DIA", - "directie": "Buy", - "tf_mare": "5min", - "tf_mic": "1min", - "calitate": "Clară", - "entry": "400.0", - "sl": "399.0", - "tp0": "400.5", - "tp1": "401.0", - "tp2": "402.0", - "risc_pct": "0.25", - "outcome_path": "TP0→TP1", - "max_reached": "TP1", - "be_moved": "True", - "pl_marius": "0.5000", - "pl_theoretical": "0.3330", - "set": "A2", - "indicator_version": "v-2026-05", - "pl_overlay_version": "marius-v1", - "csv_schema_version": "1", - "extracted_at": "2026-05-13T10:00:00Z", - "note": "", - } - base.update({k: str(v) for k, v in overrides.items()}) - return base - - -def _write_csv(path: Path, rows: list[dict[str, str]]) -> None: - path.parent.mkdir(parents=True, exist_ok=True) - with path.open("w", encoding="utf-8", newline="") as fh: - w = csv.DictWriter(fh, fieldnames=list(CSV_COLUMNS)) - w.writeheader() - for r in rows: - w.writerow({k: r.get(k, "") for k in CSV_COLUMNS}) - - -# Outcome templates (P/L values) — match scripts.pl_calc tables. -_SL = {"outcome_path": "SL", "max_reached": "SL_first", "be_moved": "False", - "pl_marius": "-1.0000", "pl_theoretical": "-1.0000"} -_TP0_SL_BE = {"outcome_path": "TP0→SL", "max_reached": "TP0", "be_moved": "True", - "pl_marius": "0.2000", "pl_theoretical": "0.1330"} -_TP0_TP1 = {"outcome_path": "TP0→TP1", "max_reached": "TP1", "be_moved": "True", - "pl_marius": "0.5000", "pl_theoretical": "0.3330"} -_TP0_TP2 = {"outcome_path": "TP0→TP2", "max_reached": "TP2", "be_moved": "True", - "pl_marius": "0.5000", "pl_theoretical": "0.6670"} -_PENDING = {"outcome_path": "pending", "max_reached": "TP0", "be_moved": "False", - "pl_marius": "", "pl_theoretical": "0.1330"} - - -def _synthetic_csv(tmp_path: Path) -> Path: - """30-trade backtest fixture. - - Set distribution: - A1: 8 rows (all closed; 3 SL, 2 TP0→SL, 2 TP0→TP1, 1 TP0→TP2) - A2: 10 rows (all closed; 4 SL, 3 TP0→SL, 2 TP0→TP1, 1 TP0→TP2) - B : 7 rows (2 pending, 5 closed; 2 SL, 2 TP0→TP1, 1 TP0→TP2) - D : 5 rows (3 pending, 2 closed; 1 SL, 1 TP0→TP1) - - Totals: n_total=30, n_pending=5, n_closed=25. - - Wins by pl_marius (>0): all TP0→SL_BE + TP0→TP1 + TP0→TP2 - A1: 2 + 2 + 1 = 5 wins / 8 - A2: 3 + 2 + 1 = 6 wins / 10 - B : 0 + 2 + 1 = 3 wins / 5 - D : 0 + 1 + 0 = 1 win / 2 - Total wins = 15 / 25 = 60.0%. - - Calitate distribution: half "Clară", half "Slabă" (alternating). - Directie distribution: 2/3 Buy, 1/3 Sell. - """ - rows: list[dict[str, str]] = [] - rid = 0 - - def add(set_label: str, outcomes: list[dict[str, str]]) -> None: - nonlocal rid - for i, outcome in enumerate(outcomes): - rid += 1 - row = _base_row( - id=rid, - screenshot_file=f"{set_label.lower()}-{rid}.png", - set=set_label, - calitate="Clară" if rid % 2 == 0 else "Slabă", - directie="Buy" if rid % 3 != 0 else "Sell", - ) - row.update({k: str(v) for k, v in outcome.items()}) - rows.append(row) - - add("A1", [_SL] * 3 + [_TP0_SL_BE] * 2 + [_TP0_TP1] * 2 + [_TP0_TP2] * 1) - add("A2", [_SL] * 4 + [_TP0_SL_BE] * 3 + [_TP0_TP1] * 2 + [_TP0_TP2] * 1) - add("B", [_PENDING] * 2 + [_SL] * 2 + [_TP0_TP1] * 2 + [_TP0_TP2] * 1) - add("D", [_PENDING] * 3 + [_SL] * 1 + [_TP0_TP1] * 1) - - path = tmp_path / "jurnal.csv" - _write_csv(path, rows) - return path - - -# --------------------------------------------------------------------------- -# compute_stats — core -# --------------------------------------------------------------------------- - - -class TestComputeStats: - def test_compute_stats_n_pending(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - assert s["n_total"] == 30 - assert s["n_pending"] == 5 - assert s["n_closed"] == 25 - - def test_compute_stats_wr_correct(self, tmp_path: Path) -> None: - """Manual win count: 15 / 25 = 60.0%.""" - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - assert s["wr"] == pytest.approx(15 / 25) - lo, hi = s["wr_ci_95"] - assert 0.0 <= lo <= s["wr"] <= hi <= 1.0 - - def test_compute_stats_per_set(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - a2 = s["per_set"]["A2"] - assert a2["n"] == 10 # 10 closed A2 trades - # A2 wins (pl_marius > 0): 3 BE + 2 TP1 + 1 TP2 = 6 / 10 - assert a2["wr"] == pytest.approx(0.60) - - def test_per_set_b_pending_excluded(self, tmp_path: Path) -> None: - """Set B has 7 total rows (2 pending + 5 closed). n must be 5.""" - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - assert s["per_set"]["B"]["n"] == 5 - # B wins: 0 BE + 2 TP1 + 1 TP2 = 3 / 5 - assert s["per_set"]["B"]["wr"] == pytest.approx(0.60) - - def test_per_directie_no_ci_keys(self, tmp_path: Path) -> None: - """per_directie omits CI fields per spec (only n / wr / expectancy).""" - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - for k, d in s["per_directie"].items(): - assert set(d.keys()) == {"n", "wr", "expectancy"}, k - - def test_overlay_theoretical_vs_marius(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - s_m = compute_stats(path, overlay="pl_marius") - s_t = compute_stats(path, overlay="pl_theoretical") - # Same N, but different expectancy. - assert s_m["n_closed"] == s_t["n_closed"] - assert s_m["expectancy"] != s_t["expectancy"] - - def test_unknown_overlay_raises(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - with pytest.raises(ValueError): - compute_stats(path, overlay="pl_imaginary") - - def test_empty_csv_no_crash(self, tmp_path: Path) -> None: - path = tmp_path / "empty.csv" - _write_csv(path, []) - s = compute_stats(path) - assert s["n_total"] == 0 - assert s["n_closed"] == 0 - assert s["per_set"] == {} - assert s["wr"] == 0.0 - assert s["wr_ci_95"] == (0.0, 0.0) - - def test_missing_csv_no_crash(self, tmp_path: Path) -> None: - # Nonexistent path: treat as empty, do not raise. - s = compute_stats(tmp_path / "ghost.csv") - assert s["n_total"] == 0 - - def test_calibration_rows_excluded(self, tmp_path: Path) -> None: - rows = [ - _base_row(id=1, source="vision", screenshot_file="v.png"), - _base_row(id=2, source="manual_calibration", screenshot_file="c.png"), - _base_row(id=3, source="vision_calibration", screenshot_file="c.png"), - ] - path = tmp_path / "j.csv" - _write_csv(path, rows) - s = compute_stats(path) - assert s["n_total"] == 1 # calibration rows filtered out - - -# --------------------------------------------------------------------------- -# render_stats -# --------------------------------------------------------------------------- - - -class TestRenderStats: - def test_render_stats_no_crash(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - s = compute_stats(path) - out = render_stats(s, "pl_marius") - assert isinstance(out, str) - assert out # non-empty - assert "STOPPING RULE" in out - - def test_render_stats_contains_sections(self, tmp_path: Path) -> None: - path = _synthetic_csv(tmp_path) - out = render_stats(compute_stats(path), "pl_marius") - for marker in ( - "Stats jurnal.csv", - "Trade-uri totale", - "GLOBAL", - "PER SET:", - "PER CALITATE", - "PER DIRECȚIE", - "DESCRIPTOR ONLY", - ): - assert marker in out, f"missing section: {marker!r}" - - def test_render_stats_flags_under_threshold(self, tmp_path: Path) -> None: - """All Sets in synthetic fixture have N<40 → all should be flagged.""" - path = _synthetic_csv(tmp_path) - out = render_stats(compute_stats(path), "pl_marius") - for k in ("A1", "A2", "B", "D"): - assert f"{k}: N=" in out - assert "NEEDS MORE DATA" in out - - def test_render_stats_empty(self, tmp_path: Path) -> None: - path = tmp_path / "empty.csv" - _write_csv(path, []) - out = render_stats(compute_stats(path), "pl_marius") - assert "Trade-uri totale: 0" in out - # No crash, no per-Set table for an empty dataset. - assert "NEEDS MORE DATA" not in out - - -# --------------------------------------------------------------------------- -# compute_calibration -# --------------------------------------------------------------------------- - - -class TestComputeCalibration: - def test_compute_calibration_pairs(self, tmp_path: Path) -> None: - rows: list[dict[str, str]] = [] - for i in range(5): - f = f"cal-{i}.png" - rows.append(_base_row( - id=i * 2 + 1, source="manual_calibration", screenshot_file=f - )) - rows.append(_base_row( - id=i * 2 + 2, source="vision_calibration", screenshot_file=f - )) - path = tmp_path / "j.csv" - _write_csv(path, rows) - cal = compute_calibration(path) - assert cal["n_pairs"] == 5 - for fld in CORE_CALIBRATION_FIELDS: - assert fld in cal["fields"] - # All identical → 5 matches, 0 mismatches per field. - assert cal["fields"][fld]["match"] == 5 - assert cal["fields"][fld]["mismatch"] == 0 - assert cal["fields"][fld]["match_rate"] == pytest.approx(1.0) - - def test_compute_calibration_mismatch_examples(self, tmp_path: Path) -> None: - """Modify entry on 2 pairs → mismatch_examples contains both.""" - rows: list[dict[str, str]] = [] - for i in range(5): - f = f"cal-{i}.png" - manual_entry = "400.0" - # First two pairs differ on entry; the rest match exactly. - vision_entry = "401.5" if i < 2 else "400.0" - rows.append(_base_row( - id=i * 2 + 1, source="manual_calibration", - screenshot_file=f, entry=manual_entry, - )) - rows.append(_base_row( - id=i * 2 + 2, source="vision_calibration", - screenshot_file=f, entry=vision_entry, - )) - path = tmp_path / "j.csv" - _write_csv(path, rows) - cal = compute_calibration(path) - assert cal["n_pairs"] == 5 - entry = cal["fields"]["entry"] - assert entry["match"] == 3 - assert entry["mismatch"] == 2 - assert entry["match_rate"] == pytest.approx(3 / 5) - assert len(entry["mismatch_examples"]) == 2 - for ex in entry["mismatch_examples"]: - assert "manual=" in ex and "vision=" in ex - - def test_calibration_examples_capped_at_3(self, tmp_path: Path) -> None: - """5 mismatches but mismatch_examples is capped at 3.""" - rows: list[dict[str, str]] = [] - for i in range(5): - f = f"cal-{i}.png" - rows.append(_base_row( - id=i * 2 + 1, source="manual_calibration", - screenshot_file=f, entry="400.0", - )) - rows.append(_base_row( - id=i * 2 + 2, source="vision_calibration", - screenshot_file=f, entry="500.0", - )) - path = tmp_path / "j.csv" - _write_csv(path, rows) - cal = compute_calibration(path) - assert cal["fields"]["entry"]["mismatch"] == 5 - assert len(cal["fields"]["entry"]["mismatch_examples"]) == 3 - - def test_calibration_numeric_tolerance(self, tmp_path: Path) -> None: - """Floats within 0.01 must NOT count as a mismatch.""" - rows = [ - _base_row( - id=1, source="manual_calibration", - screenshot_file="cal-1.png", entry="400.005", - ), - _base_row( - id=2, source="vision_calibration", - screenshot_file="cal-1.png", entry="400.010", - ), - ] - path = tmp_path / "j.csv" - _write_csv(path, rows) - cal = compute_calibration(path) - assert cal["fields"]["entry"]["match"] == 1 - assert cal["fields"]["entry"]["mismatch"] == 0 - - def test_calibration_outside_tolerance(self, tmp_path: Path) -> None: - """Floats > 0.01 apart DO count as a mismatch.""" - rows = [ - _base_row( - id=1, source="manual_calibration", - screenshot_file="cal-1.png", entry="400.00", - ), - _base_row( - id=2, source="vision_calibration", - screenshot_file="cal-1.png", entry="400.05", - ), - ] - path = tmp_path / "j.csv" - _write_csv(path, rows) - cal = compute_calibration(path) - assert cal["fields"]["entry"]["mismatch"] == 1 - - def test_calibration_no_pairs(self, tmp_path: Path) -> None: - """No paired screenshot → n_pairs=0, all rates 0.0.""" - path = tmp_path / "j.csv" - _write_csv(path, [ - _base_row(id=1, source="manual_calibration", screenshot_file="lonely.png"), - ]) - cal = compute_calibration(path) - assert cal["n_pairs"] == 0 - for fld in CORE_CALIBRATION_FIELDS: - assert cal["fields"][fld]["match"] == 0 - assert cal["fields"][fld]["mismatch"] == 0 - - def test_render_calibration_no_crash(self, tmp_path: Path) -> None: - rows = [ - _base_row(id=1, source="manual_calibration", - screenshot_file="cal-1.png", directie="Buy"), - _base_row(id=2, source="vision_calibration", - screenshot_file="cal-1.png", directie="Sell", - entry="400.0", sl="401.0", tp0="399.5", - tp1="399.0", tp2="398.0"), - ] - path = tmp_path / "j.csv" - _write_csv(path, rows) - out = render_calibration(compute_calibration(path)) - assert "Calibration P4" in out - assert "directie" in out - - def test_render_calibration_empty(self, tmp_path: Path) -> None: - path = tmp_path / "empty.csv" - _write_csv(path, []) - out = render_calibration(compute_calibration(path)) - assert "0" in out - assert "FAIL" not in out - assert "PASS" not in out - - -# --------------------------------------------------------------------------- -# CLI -# --------------------------------------------------------------------------- - - -class TestCLI: - def test_main_stats( - self, tmp_path: Path, capsys: pytest.CaptureFixture - ) -> None: - path = _synthetic_csv(tmp_path) - rc = main(["--csv", str(path)]) - assert rc == 0 - assert "Stats jurnal.csv" in capsys.readouterr().out - - def test_main_overlay( - self, tmp_path: Path, capsys: pytest.CaptureFixture - ) -> None: - path = _synthetic_csv(tmp_path) - rc = main(["--csv", str(path), "--overlay", "pl_theoretical"]) - assert rc == 0 - assert "pl_theoretical" in capsys.readouterr().out - - def test_main_calibration( - self, tmp_path: Path, capsys: pytest.CaptureFixture - ) -> None: - rows = [ - _base_row(id=1, source="manual_calibration", - screenshot_file="cal-1.png"), - _base_row(id=2, source="vision_calibration", - screenshot_file="cal-1.png"), - ] - path = tmp_path / "j.csv" - _write_csv(path, rows) - rc = main(["--csv", str(path), "--calibration"]) - assert rc == 0 - out = capsys.readouterr().out - assert "Calibration P4" in out - assert "PASS" in out diff --git a/tests/test_stats_ci.py b/tests/test_stats_ci.py deleted file mode 100644 index 12d5367..0000000 --- a/tests/test_stats_ci.py +++ /dev/null @@ -1,83 +0,0 @@ -"""Pure-math tests for stats CI primitives (no I/O).""" - -from __future__ import annotations - -import sys -from pathlib import Path - -import pytest - -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) - -from scripts.stats import bootstrap_expectancy_ci, wilson_ci # noqa: E402 - - -# --------------------------------------------------------------------------- -# Wilson CI -# --------------------------------------------------------------------------- - - -class TestWilsonCI: - def test_wilson_n_zero(self) -> None: - assert wilson_ci(0, 0) == (0.0, 0.0) - - def test_wilson_perfect_winrate(self) -> None: - lo, hi = wilson_ci(10, 10) - assert lo > 0.65 - assert hi == pytest.approx(1.0, abs=1e-12) - - def test_wilson_reference_15_55(self) -> None: - """wins=8, n=15 (WR≈53%) → CI approximately [29%, 76%] ±2%.""" - lo, hi = wilson_ci(8, 15) - assert lo == pytest.approx(0.29, abs=0.02) - assert hi == pytest.approx(0.76, abs=0.02) - - def test_wilson_all_losses(self) -> None: - lo, hi = wilson_ci(0, 10) - assert lo == pytest.approx(0.0, abs=1e-12) - assert hi < 0.35 - - def test_wilson_wins_out_of_range(self) -> None: - with pytest.raises(ValueError): - wilson_ci(11, 10) - with pytest.raises(ValueError): - wilson_ci(-1, 10) - - def test_wilson_clamps_at_50pct_n40(self) -> None: - """Reference at WR=50%, N=40: CI ≈ [35.2%, 64.8%].""" - lo, hi = wilson_ci(20, 40) - assert lo == pytest.approx(0.352, abs=0.005) - assert hi == pytest.approx(0.648, abs=0.005) - - -# --------------------------------------------------------------------------- -# Bootstrap CI -# --------------------------------------------------------------------------- - - -class TestBootstrap: - def test_bootstrap_deterministic(self) -> None: - values = [1.0, -0.5, 0.5, -1.0] - a = bootstrap_expectancy_ci(values, n_resamples=1000, seed=42) - b = bootstrap_expectancy_ci(values, n_resamples=1000, seed=42) - assert a == b - - def test_bootstrap_different_seed_different_result(self) -> None: - values = [1.0, -0.5, 0.5, -1.0, 0.2, -0.3, 0.5] - a = bootstrap_expectancy_ci(values, n_resamples=1000, seed=1) - b = bootstrap_expectancy_ci(values, n_resamples=1000, seed=2) - assert a != b - - def test_bootstrap_empty(self) -> None: - assert bootstrap_expectancy_ci([], n_resamples=100, seed=0) == (0.0, 0.0) - - def test_bootstrap_single_value(self) -> None: - lo, hi = bootstrap_expectancy_ci([0.5], n_resamples=100, seed=0) - assert lo == pytest.approx(0.5, abs=1e-9) - assert hi == pytest.approx(0.5, abs=1e-9) - - def test_bootstrap_brackets_the_mean(self) -> None: - values = [0.5, -1.0, 0.5, 0.5, -1.0, 0.2, -0.3, 0.5, -1.0, 0.5] * 5 - mean = sum(values) / len(values) - lo, hi = bootstrap_expectancy_ci(values, n_resamples=1000, seed=7) - assert lo <= mean <= hi diff --git a/tests/test_vision_schema.py b/tests/test_vision_schema.py deleted file mode 100644 index 08368f0..0000000 --- a/tests/test_vision_schema.py +++ /dev/null @@ -1,225 +0,0 @@ -"""Tests for scripts.vision_schema.M2DExtraction.""" -from __future__ import annotations - -import json -import sys -from datetime import datetime, timezone -from pathlib import Path - -import pytest -from pydantic import ValidationError - -sys.path.insert(0, str(Path(__file__).resolve().parents[1])) - -from scripts.vision_schema import ( # noqa: E402 - M2DExtraction, - parse_extraction, - parse_extraction_dict, -) - - -def _buy_payload(**overrides) -> dict: - base = { - "screenshot_file": "dia-1min-example.png", - "data": "2026-05-13", - "ora_utc": "14:23", - "instrument": "DIA", - "directie": "Buy", - "tf_mare": "5min", - "tf_mic": "1min", - "calitate": "Clară", - "entry": 400.0, - "sl": 399.0, - "tp0": 400.5, - "tp1": 401.0, - "tp2": 402.0, - "risc_pct": 0.25, - "outcome_path": "TP0→TP1", - "max_reached": "TP1", - "be_moved": True, - "confidence": "high", - "ambiguities": [], - "note": "", - } - base.update(overrides) - return base - - -def _sell_payload(**overrides) -> dict: - base = { - "screenshot_file": "dia-sell.png", - "data": "2026-05-13", - "ora_utc": "15:00", - "instrument": "US30", - "directie": "Sell", - "tf_mare": "15min", - "tf_mic": "3min", - "calitate": "Mai mare ca impuls", - "entry": 400.0, - "sl": 401.0, - "tp0": 399.5, - "tp1": 399.0, - "tp2": 398.0, - "risc_pct": 0.3, - "outcome_path": "TP0→TP2", - "max_reached": "TP2", - "be_moved": False, - "confidence": "medium", - "ambiguities": ["entry overlap with wick"], - "note": "nothing", - } - base.update(overrides) - return base - - -def test_happy_path_buy(): - m = parse_extraction_dict(_buy_payload()) - assert m.directie == "Buy" - assert m.entry == 400.0 - - -def test_happy_path_sell(): - m = parse_extraction_dict(_sell_payload()) - assert m.directie == "Sell" - assert m.sl > m.entry > m.tp0 > m.tp1 > m.tp2 - - -def test_parse_extraction_from_json_str(): - payload = _buy_payload() - m = parse_extraction(json.dumps(payload)) - assert isinstance(m, M2DExtraction) - - -@pytest.mark.parametrize( - "field,bad_value", - [ - ("directie", "Long"), - ("instrument", "SPY"), - ("tf_mare", "30min"), - ("tf_mic", "2min"), - ("calitate", "Bună"), - ("outcome_path", "BE"), - ("max_reached", "BE"), - ("confidence", "very-high"), - ], -) -def test_each_literal_rejection(field, bad_value): - payload = _buy_payload(**{field: bad_value}) - with pytest.raises(ValidationError): - parse_extraction_dict(payload) - - -def test_entry_equals_sl(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(entry=399.0, sl=399.0)) - - -def test_buy_tp_inverted(): - # tp1 < tp0 violates ordering - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(tp0=401.0, tp1=400.5, tp2=402.0)) - - -def test_buy_sl_above_entry_rejected(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(sl=400.5)) - - -def test_sell_order_correct(): - m = parse_extraction_dict(_sell_payload()) - assert m.sl > m.entry - assert m.entry > m.tp0 - assert m.tp0 > m.tp1 > m.tp2 - - -def test_sell_order_inverted_rejected(): - # using Buy-ordering values with directie=Sell - with pytest.raises(ValidationError): - parse_extraction_dict( - _sell_payload(sl=399.0, entry=400.0, tp0=400.5, tp1=401.0, tp2=402.0) - ) - - -def test_data_in_future(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(data="2099-01-01")) - - -def test_data_today_ok(): - today = datetime.now(timezone.utc).date().isoformat() - m = parse_extraction_dict(_buy_payload(data=today)) - assert m.data == today - - -def test_outcome_path_sl_max_reached_inconsistent(): - with pytest.raises(ValidationError): - parse_extraction_dict( - _buy_payload(outcome_path="SL", max_reached="TP1") - ) - - -def test_outcome_path_sl_max_reached_sl_first_ok(): - m = parse_extraction_dict( - _buy_payload(outcome_path="SL", max_reached="SL_first") - ) - assert m.outcome_path == "SL" - - -def test_outcome_path_tp0_max_reached_sl_first_rejected(): - with pytest.raises(ValidationError): - parse_extraction_dict( - _buy_payload(outcome_path="TP0→TP1", max_reached="SL_first") - ) - - -def test_outcome_path_pending_any_max_reached_ok(): - m = parse_extraction_dict( - _buy_payload(outcome_path="pending", max_reached="SL_first") - ) - assert m.outcome_path == "pending" - - -def test_extra_field_forbidden(): - payload = _buy_payload() - payload["unexpected_field"] = "x" - with pytest.raises(ValidationError): - parse_extraction_dict(payload) - - -def test_data_bad_format(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(data="2026/05/13")) - - -def test_data_bad_format_short(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(data="26-05-13")) - - -def test_ora_utc_bad_format(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(ora_utc="14:23:00")) - - -def test_ora_utc_bad_format_no_colon(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(ora_utc="1423")) - - -def test_ora_utc_invalid_hour(): - with pytest.raises(ValidationError): - parse_extraction_dict(_buy_payload(ora_utc="25:00")) - - -def test_ambiguities_default_empty(): - payload = _buy_payload() - del payload["ambiguities"] - m = parse_extraction_dict(payload) - assert m.ambiguities == [] - - -def test_note_default_empty(): - payload = _buy_payload() - del payload["note"] - m = parse_extraction_dict(payload) - assert m.note == ""