feat: calibration/ corpus + scenarii regresie FSM

- calibration/frames/: 16 PNG-uri ground-truth numite {ts}_{color}.png,
  copiate din logs/fires (izolate de samples/ și logs/fires/ care se pot goli)
- calibration/calibration_labels.json: mutat din samples/, curățat de entries
  cu fișiere inexistente, extins la acoperire completă 7 culori → 16/16 PASS
- calibration/scenarios.json: 8 secvențe FSM (BUY/SELL full cycle, phase_skip,
  catchup, post-fire suppression) pe frame-uri reale
- tests/test_scenarios_regression.py: parametrizat pe scenarios.json, asertează
  color+state+reason+trigger+alerts+scheduler prin pipeline-ul
  Detector → _handle_tick
- docs: README + CLAUDE reflectă noua structură, incidentul 2026-04-20/21
  (pixel saturat UNKNOWN → FSM blocat în PRIMED → polling continuu) +
  troubleshooting pentru trigger UNKNOWN

Pytest: 184 → 192 passed (+8 scenarii regresie).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-21 08:32:11 +03:00
parent bed79fcc35
commit 9e8cbafbd4
24 changed files with 578 additions and 78 deletions

49
calibration/README.md Normal file
View File

@@ -0,0 +1,49 @@
# calibration/ — frame corpus for validation & regression
Two artifacts, one frame pool:
- `calibration_labels.json` — per-frame color labels. Used by
`atm validate-calibration` to check the current palette classifies known-good
dots correctly before a live session.
- `scenarios.json` — ordered frame sequences per FSM scenario (full cycle,
phase skip, catchup, post-fire suppression). Consumed by
`tests/test_scenarios_regression.py` which runs each sequence through the
full `Detector → _handle_tick` pipeline and asserts color, FSM reason/state,
emitted alerts, and scheduler on/off.
Frames live in `calibration/frames/` and are self-contained: purging
`logs/fires/` or `samples/` does not break either artifact.
## calibration_labels.json schema
## Schema
A JSON array of entries. Each entry:
| Field | Type | Required | Description |
|------------|---------|----------|----------------------------------------------------------------|
| `path` | string | yes | Path to a PNG frame (relative to CWD or absolute). |
| `expected` | string | yes | Expected color name: one of `turquoise`, `yellow`, `dark_green`, `dark_red`, `light_green`, `light_red`, `gray`. |
| `note` | string | no | Freeform annotation; shown in SUGGESTIONS output. |
## Usage
```bash
atm validate-calibration calibration/calibration_labels.json
```
Exit codes:
- `0` — every sample PASS
- `1` — one or more FAIL
- `2` — label file missing or malformed JSON
## Adding new samples
1. Find a screenshot in `logs/fires/` whose dot color you can verify by eye.
2. **Copy it into `calibration/frames/`** — this directory is self-contained so
`logs/fires/` and `samples/` can be emptied without breaking validation.
3. Append an entry with `path` (pointing to `calibration/frames/...`),
`expected`, and an optional `note`.
4. Re-run validation. If it FAILs, the SUGGESTIONS section will tell you the
RGB distance between the observed pixel and the expected color's center —
use that as input for `atm calibrate`.

View File

@@ -0,0 +1,82 @@
[
{
"path": "calibration/frames/20260420_200002_turquoise.png",
"expected": "turquoise",
"note": "BUY arm visible in poll; rgb=(0,253,253)"
},
{
"path": "calibration/frames/20260421_072757_turquoise.png",
"expected": "turquoise",
"note": "BUY arm via manual /ss; rgb=(0,253,253)"
},
{
"path": "calibration/frames/20260420_171501_yellow.png",
"expected": "yellow",
"note": "SELL arm event; rgb=(253,253,0)"
},
{
"path": "calibration/frames/20260420_194505_yellow.png",
"expected": "yellow",
"note": "SELL arm event; rgb=(253,253,0)"
},
{
"path": "calibration/frames/20260420_194721_yellow.png",
"expected": "yellow",
"note": "SELL arm visible in manual /ss; rgb=(253,253,0)"
},
{
"path": "calibration/frames/20260418_124645_dark_green.png",
"expected": "dark_green",
"note": "BUY prime catchup; rgb=(0,128,0)"
},
{
"path": "calibration/frames/20260420_185102_dark_green.png",
"expected": "dark_green",
"note": "BUY prime; rgb=(0,128,0)"
},
{
"path": "calibration/frames/20260420_213706_dark_green.png",
"expected": "dark_green",
"note": "BUY prime catchup; rgb=(0,128,0)"
},
{
"path": "calibration/frames/20260420_172104_dark_red.png",
"expected": "dark_red",
"note": "SELL prime; rgb=(128,0,0)"
},
{
"path": "calibration/frames/20260420_195701_dark_red.png",
"expected": "dark_red",
"note": "SELL prime; rgb=(128,0,0)"
},
{
"path": "calibration/frames/20260420_210905_dark_red.png",
"expected": "dark_red",
"note": "SELL prime; rgb=(128,0,0)"
},
{
"path": "calibration/frames/20260420_163303_light_green.png",
"expected": "light_green",
"note": "BUY trigger (FIRE); rgb=(0,255,0)"
},
{
"path": "calibration/frames/20260420_214908_light_green.png",
"expected": "light_green",
"note": "regression 2026-04-20: BUY trigger visible in poll (original complaint); rgb=(0,255,0) was UNKNOWN under pre-2026-04-21 calibration"
},
{
"path": "calibration/frames/20260420_173004_light_red.png",
"expected": "light_red",
"note": "SELL trigger (FIRE); rgb=(255,0,0)"
},
{
"path": "calibration/frames/20260420_175005_gray.png",
"expected": "gray",
"note": "idle gray dot via manual /ss; rgb=(128,128,128)"
},
{
"path": "calibration/frames/20260420_185702_gray.png",
"expected": "gray",
"note": "idle gray dot in poll; rgb=(128,128,128)"
}
]

Binary file not shown.

After

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 348 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 347 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 342 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 299 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 288 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 297 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 302 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 285 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 317 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 274 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 252 KiB

248
calibration/scenarios.json Normal file
View File

@@ -0,0 +1,248 @@
[
{
"id": "buy_full_cycle",
"description": "IDLE → ARMED_BUY → PRIMED_BUY → IDLE(fire). Turquoise arm, dark_green prime, light_green trigger.",
"steps": [
{
"frame": "calibration/frames/20260420_200002_turquoise.png",
"expected_color": "turquoise",
"expected_reason": "arm",
"expected_state": "ARMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_185102_dark_green.png",
"expected_color": "dark_green",
"expected_reason": "prime",
"expected_state": "PRIMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["prime"],
"expected_scheduler_running": true
},
{
"frame": "calibration/frames/20260420_163303_light_green.png",
"expected_color": "light_green",
"expected_reason": "fire",
"expected_state": "IDLE",
"expected_trigger": "BUY",
"expected_new_alerts": [],
"expected_scheduler_running": false
}
]
},
{
"id": "sell_full_cycle",
"description": "Mirror of buy_full_cycle: yellow arm, dark_red prime, light_red trigger.",
"steps": [
{
"frame": "calibration/frames/20260420_171501_yellow.png",
"expected_color": "yellow",
"expected_reason": "arm",
"expected_state": "ARMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_172104_dark_red.png",
"expected_color": "dark_red",
"expected_reason": "prime",
"expected_state": "PRIMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["prime"],
"expected_scheduler_running": true
},
{
"frame": "calibration/frames/20260420_173004_light_red.png",
"expected_color": "light_red",
"expected_reason": "fire",
"expected_state": "IDLE",
"expected_trigger": "SELL",
"expected_new_alerts": [],
"expected_scheduler_running": false
}
]
},
{
"id": "buy_phase_skip",
"description": "ARMED_BUY → light_green direct (dark_green missed). Backstop `fire_on_phase_skip` emits phase_skip_fire alert.",
"steps": [
{
"frame": "calibration/frames/20260421_072757_turquoise.png",
"expected_color": "turquoise",
"expected_reason": "arm",
"expected_state": "ARMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_214908_light_green.png",
"expected_color": "light_green",
"expected_reason": "phase_skip",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": ["phase_skip_fire"],
"expected_scheduler_running": false
}
]
},
{
"id": "sell_phase_skip",
"description": "Mirror: ARMED_SELL → light_red direct (dark_red missed).",
"steps": [
{
"frame": "calibration/frames/20260420_194505_yellow.png",
"expected_color": "yellow",
"expected_reason": "arm",
"expected_state": "ARMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_173004_light_red.png",
"expected_color": "light_red",
"expected_reason": "phase_skip",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": ["phase_skip_fire"],
"expected_scheduler_running": false
}
]
},
{
"id": "buy_catchup",
"description": "Start with dark_green in IDLE (no arm observed). Catchup synth-feeds turquoise → emits arm+prime alerts. FSM ends in PRIMED_BUY.",
"steps": [
{
"frame": "calibration/frames/20260418_124645_dark_green.png",
"expected_color": "dark_green",
"expected_reason": "prime",
"expected_state": "PRIMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["arm", "prime"],
"expected_scheduler_running": true
}
]
},
{
"id": "sell_catchup",
"description": "Mirror: start with dark_red in IDLE. Catchup synth-yellow → arm+prime alerts.",
"steps": [
{
"frame": "calibration/frames/20260420_195701_dark_red.png",
"expected_color": "dark_red",
"expected_reason": "prime",
"expected_state": "PRIMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["arm", "prime"],
"expected_scheduler_running": true
}
]
},
{
"id": "buy_post_fire_suppression",
"description": "After BUY fire, residual dark_green in IDLE must NOT re-prime. User rule: new arming (turquoise) required before priming alerts become valid again.",
"steps": [
{
"frame": "calibration/frames/20260420_200002_turquoise.png",
"expected_color": "turquoise",
"expected_reason": "arm",
"expected_state": "ARMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_185102_dark_green.png",
"expected_color": "dark_green",
"expected_reason": "prime",
"expected_state": "PRIMED_BUY",
"expected_trigger": null,
"expected_new_alerts": ["prime"],
"expected_scheduler_running": true
},
{
"frame": "calibration/frames/20260420_163303_light_green.png",
"expected_color": "light_green",
"expected_reason": "fire",
"expected_state": "IDLE",
"expected_trigger": "BUY",
"expected_new_alerts": [],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_213706_dark_green.png",
"expected_color": "dark_green",
"expected_reason": "noise",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": [],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260418_124645_dark_green.png",
"expected_color": "dark_green",
"expected_reason": "noise",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": [],
"expected_scheduler_running": false
}
]
},
{
"id": "sell_post_fire_suppression",
"description": "Mirror: after SELL fire, residual dark_red must NOT re-prime until new yellow arming.",
"steps": [
{
"frame": "calibration/frames/20260420_171501_yellow.png",
"expected_color": "yellow",
"expected_reason": "arm",
"expected_state": "ARMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["arm"],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_172104_dark_red.png",
"expected_color": "dark_red",
"expected_reason": "prime",
"expected_state": "PRIMED_SELL",
"expected_trigger": null,
"expected_new_alerts": ["prime"],
"expected_scheduler_running": true
},
{
"frame": "calibration/frames/20260420_173004_light_red.png",
"expected_color": "light_red",
"expected_reason": "fire",
"expected_state": "IDLE",
"expected_trigger": "SELL",
"expected_new_alerts": [],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_195701_dark_red.png",
"expected_color": "dark_red",
"expected_reason": "noise",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": [],
"expected_scheduler_running": false
},
{
"frame": "calibration/frames/20260420_210905_dark_red.png",
"expected_color": "dark_red",
"expected_reason": "noise",
"expected_state": "IDLE",
"expected_trigger": null,
"expected_new_alerts": [],
"expected_scheduler_running": false
}
]
}
]