feat(voice): polish voice loop UX — filler kill, barge-in, DTX flush, time/RO TTS
End-to-end voice UX iteration after DAVE E2E shipped. Each change addresses a
real symptom Marius hit in live testing today:
- Kill the 3s filler ("mă gândesc"): Claude p50 is 4-7s so the filler always
fired BEFORE the response and collided with it. Removed all filler infra
from pipeline.py + tts_stream.py (FILLER_DELAY_S, _filler_task, push_filler,
load_thinking_wav, thinking.wav cache).
- Barge-in: ttsq.clear() at the top of on_segment_done drops stale frames so
a new utterance cuts off Echo's previous response cleanly.
- DTX silence flush: Discord stops sending RTP packets when the user goes
silent (DTX), so the inline silence-check in sink.write() never fired for
the trailing audio of an utterance — STT was missed entirely. Added a
background poller thread that checks the silence-flush condition every
200ms independent of incoming packets.
- Discord audio cadence fix: EchoStreamingAudioSource.read() blocked 100ms
per call when pcm_queue was empty, wrecking Discord's 20ms frame pacing →
client interpreted the stream as stutter and discarded leading frames
(Marius heard "4 de minute în București" instead of the full sentence).
Switched to get_frame_nowait() — instant return, silence frame on empty.
- RO time expansion: "23:09" was being read as "douăzeci și trei:nouă"
with literal colon. Added expand_time() with feminine-correct minute
formatting (un minut / două minute / douăzeci de minute / una de minute).
- Supertonic Unicode sanitize centralized in tools/tts.py: Romanian curly
quotes (`„`, `"`, `"`, `—`, `…`) crash Supertonic with HTTP 500. Map them
to ASCII at the synthesize() entry so BOTH voice mode and /audio command
are covered without duplication. normalize.py re-exports for compat.
- Whisper offline: WhisperModel(..., local_files_only=True) — no more
huggingface.co metadata GET on every startup. Model is already cached.
- Diagnostic logging across the chain: sink first-packet, VAD first-speech,
voice stream block (Claude → callback), push_text (text → clauses queued),
TTS pushed (clauses → frames). Lets future "spoke but Echo silent" bugs
pinpoint exactly where the chain breaks.
- Captured Supertonic curly-quote lesson in tasks/lessons.md.
All 76 voice tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,13 @@ Lecții capturate din corectările lui Marius. Citește acest fișier la începu
|
||||
|
||||
<!-- Lecțiile se adaugă mai jos, cele mai noi sus. -->
|
||||
|
||||
## Supertonic rejectează ghilimelele curly (Unicode) cu HTTP 500
|
||||
**Data:** 2026-05-27
|
||||
**Context:** Marius a dat o comandă audio pe Discord cu un URL, iar răspunsul lui Claude conținea `„foo"` (ghilimele românești curly). Supertonic a returnat `HTTP 500: synthesis failed: Found 1 unsupported character(s): ['„']` și răspunsul nu s-a mai auzit. Fără retry logic vizibil în UX — pur și simplu tace.
|
||||
**Greșeala:** Am presupus că `normalize_for_tts` produce text deja "TTS-safe" pentru Supertonic. În realitate `strip_markdown` păstrează ghilimelele Unicode (`„` U+201E, `"` U+201D, `—` U+2014, `…` U+2026, etc.) pe care Supertonic le refuză.
|
||||
**Regula:** Înainte de orice apel HTTP la Supertonic, **sanitizează punctuația Unicode** la echivalentele ASCII (`„` `"` `"` → `"`, `'` `'` `‚` → `'`, `–` `—` → `-`, `…` → `...`, `«` `»` → `"`). Funcția `sanitize_punctuation` în `src/voice/normalize.py` face asta și e apelată chiar după `strip_markdown` în pipeline. Dacă apar caractere noi care crapă Supertonic (ex: simboluri matematice, săgeți), adaugă-le în `_TTS_PUNCT_MAP`.
|
||||
**Când se aplică:** Orice cod care trimite text la Supertonic (`tools/tts.py`, `src/voice/tts_stream.py`). Inclusiv testare manuală cu `curl` — folosește text românesc realistic (include `„foo"`, em-dash `—`, ellipsis `…`).
|
||||
|
||||
## Mai multe threads ≠ mai rapid — fitează `cpu_threads` pe physical cores, nu logical
|
||||
**Data:** 2026-05-27
|
||||
**Context:** Benchmark `tools/voice_bench.py` pentru faster-whisper `small` int8 pe i7-6700T (4 physical / 8 logical cores). Marius a urcat VM-ul de la 2 → 4 → 6 cores online, așteptând că mai multe = mai rapid.
|
||||
|
||||
Reference in New Issue
Block a user