feat(voice): DAVE E2E + full voice UX (squash of voice/dave-recv)
Squashed branch: voice/dave-recv → master. Closes Pas 12 (DAVE E2E) and lands voice-mode UX polish + verbal voice control on top of the Pas 1-10 scaffolding already on master. ## DAVE E2E receive-side decrypt (e4f3177) Vendored fork: discord-ext-voice-recv 0.5.3a+echo.dave1. Patches the receive pipeline to handle Discord's mandatory DAVE encryption on voice gateway v=8. - `_maybe_dave_decrypt`: uses davey.can_passthrough(user_id) as primary gate, falls through to dave.decrypt for DAVE-epoch peers, drops on decrypt failure without killing the reader thread. - VAD fix: silero-vad v5+ requires exactly 512 samples; our 100ms window (1600 samples) was silently raising ValueError → STT never fired. Now slice into 512-sample chunks. - Whisper: bumped beam_size 1→5 and added RO initial_prompt. - Tests: 11 DAVE unit tests + 2 callback integration tests + contract test with fork-version guard. ## Voice UX polish (d1bc77e) - Killed the 3s "mă gândesc" filler (always collided with Claude p50 4-7s). - Barge-in via `ttsq.clear()` at top of `on_segment_done`. - DTX silence-flush poller (200ms tick) — Discord stops sending RTP packets when silent, so the inline silence-check in sink.write() never fired for trailing audio; background thread handles it. - `EchoStreamingAudioSource.read()` non-blocking — old `get_frame(timeout=0.1)` wrecked Discord's 20ms cadence and the client interpreted bursts as stuttering (Marius heard "4 de minute" instead of full sentence). - RO time expansion: 23:09 → "douăzeci și trei și nouă minute". - Supertonic Unicode sanitize centralized in tools/tts.py. - Whisper local_files_only=True — no HF metadata GET on each startup. - Diagnostic logging through sink → VAD → Claude stream → TTS chain. ## Voice mode iteration (e589e48) - `personality/VOICE_MODE.md` — voice-tailored system prompt (short, no markdown, no abbreviations, time without seconds, distances in "mii"/"milioane"); plumbed via build_system_prompt(voice_mode=True). - Isolated voice session key `voice:<channel_id>` — voice doesn't share context with text adapter on the same channel; auto-applied without /clear ceremony. /clear drops both keys. - Metric units + Romanian thousands (normalize.py): "384.000 km" → "trei sute optzeci și patru de mii de kilometri" with feminine-correct pluralization and "de" particle for ≥20. - `/voice setvoice <M1-F5>` slash command with native autocomplete; swaps live + persists voice.default_voice to config.json. - Verbal voice change (src/voice/voice_commands.py + 29 tests) — "schimbă vocea pe M5", "voce em cinci", with permissive substring fallback for Whisper-mangled forms like "Mâcinci"=M5 and "unul cinci"=M5. Whisper initial_prompt now lists voice vocabulary to bias STT toward clean outputs. - Fast barge-in: VAD ≥2 consecutive windows (~200ms) on Marius's user while Echo has pending TTS frames → cut him off mid-sentence so user doesn't wait the full silence + STT cycle. Acoustic echo bleed-through still requires headphones (no AEC). ## Test suite 130 voice + router tests pass (test_voice_recv_dave, test_voice_session_cleanup, test_voice_adapter_contract, test_voice_normalize, test_voice_commands, test_router). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
118
src/voice/voice_commands.py
Normal file
118
src/voice/voice_commands.py
Normal file
@@ -0,0 +1,118 @@
|
||||
"""Detect in-band voice commands from STT transcripts.
|
||||
|
||||
The voice pipeline transcribes Marius's speech via Whisper and dispatches the
|
||||
text to Claude. Some utterances are not questions for Claude — they're
|
||||
control commands for the voice stack itself. This module parses those out
|
||||
*before* the Claude round-trip so they take effect instantly and don't waste
|
||||
a Claude session turn.
|
||||
|
||||
Currently handled:
|
||||
* change TTS voice — "schimbă vocea pe M5", "vorbește cu vocea F3",
|
||||
"voce em cinci", "voce feminină 3", etc.
|
||||
|
||||
The parser is intentionally conservative: it requires BOTH a voice trigger
|
||||
word ("voce", "vorbește", "schimbă", "treci pe") AND a recognizable voice
|
||||
ID. A bare "M5" without context is NOT a command — Marius might be quoting
|
||||
a string.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Optional
|
||||
|
||||
|
||||
_VALID_VOICES = {f"M{i}" for i in range(1, 6)} | {f"F{i}" for i in range(1, 6)}
|
||||
|
||||
|
||||
# Trigger words that suggest the user is talking ABOUT the voice, not just
|
||||
# saying something that happens to contain a voice-ID-looking substring.
|
||||
_VOICE_TRIGGER_RE = re.compile(
|
||||
r'\b(voce|vocea|voci|voice|vorbe[șs]te|schimb[aăÎ]|treci\s+pe)\b',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# Direct form: "M5", "F 3", "m5", etc.
|
||||
_VOICE_ID_DIRECT_RE = re.compile(
|
||||
r'\b([MF])\s*([1-5])\b',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
# Word form: "em cinci", "M trei", "masculin doi", "feminină patru", etc.
|
||||
# Whisper often transcribes "M5" as "em cinci" / "M cinci" because letter
|
||||
# names are spelled out phonetically in Romanian.
|
||||
_VOICE_ID_WORDS_RE = re.compile(
|
||||
r'\b(em|m|masculin[aăe]?|ef|f|feminin[aăe]?)\s+(unu|una|doi|dou[ăa]|trei|patru|cinci|[1-5])\b',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
|
||||
_DIGIT_WORD_TO_INT = {
|
||||
'unu': 1, 'una': 1, 'unul': 1, '1': 1,
|
||||
'doi': 2, 'două': 2, 'doua': 2, '2': 2,
|
||||
'trei': 3, '3': 3,
|
||||
'patru': 4, '4': 4,
|
||||
'cinci': 5, '5': 5,
|
||||
}
|
||||
|
||||
# Substring fallback: matches digit roots even when Whisper glues them into
|
||||
# compound non-words like "Mâcinci" (for "M cinci"=M5).
|
||||
_DIGIT_SUBSTR_RE = re.compile(
|
||||
r'(cinci|patru|trei|dou[ăa]|unul|unu|una)',
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
_F_GENDER_HINT_RE = re.compile(r'feminin|\bef\b|\bF\d?\b', re.IGNORECASE)
|
||||
|
||||
|
||||
def _normalize_gender(word: str) -> Optional[str]:
|
||||
"""Map gender word to 'M' or 'F'."""
|
||||
w = word.lower()
|
||||
if w in ('m', 'em') or w.startswith('masculin'):
|
||||
return 'M'
|
||||
if w in ('f', 'ef') or w.startswith('feminin'):
|
||||
return 'F'
|
||||
return None
|
||||
|
||||
|
||||
def detect_voice_change(text: str) -> Optional[str]:
|
||||
"""Parse a transcript for a 'change voice' command.
|
||||
|
||||
Returns the target voice id (one of M1-M5, F1-F5) or None if no command
|
||||
was detected. Requires both a voice trigger word and a voice ID.
|
||||
"""
|
||||
if not text:
|
||||
return None
|
||||
if not _VOICE_TRIGGER_RE.search(text):
|
||||
return None
|
||||
# Try the direct form first (M5, F3, etc.)
|
||||
m = _VOICE_ID_DIRECT_RE.search(text)
|
||||
if m:
|
||||
candidate = f"{m.group(1).upper()}{m.group(2)}"
|
||||
if candidate in _VALID_VOICES:
|
||||
return candidate
|
||||
# Fall back to the word form ("em cinci", "feminin trei", ...).
|
||||
m = _VOICE_ID_WORDS_RE.search(text)
|
||||
if m:
|
||||
gender = _normalize_gender(m.group(1))
|
||||
digit = _DIGIT_WORD_TO_INT.get(m.group(2).lower())
|
||||
if gender is not None and digit is not None:
|
||||
candidate = f"{gender}{digit}"
|
||||
if candidate in _VALID_VOICES:
|
||||
return candidate
|
||||
# Permissive fallback: Whisper sometimes glues the letter into the next
|
||||
# word ("Mâcinci" for "M cinci") or replaces it ("unul cinci" for
|
||||
# "M unu cinci"). After a voice trigger word, scan for any digit-word
|
||||
# substring and infer gender (F if a feminine marker is present, else M).
|
||||
digit_hits = _DIGIT_SUBSTR_RE.findall(text)
|
||||
digits = [_DIGIT_WORD_TO_INT[d.lower()] for d in digit_hits
|
||||
if d.lower() in _DIGIT_WORD_TO_INT]
|
||||
digits = [d for d in digits if 1 <= d <= 5]
|
||||
if digits:
|
||||
gender = 'F' if _F_GENDER_HINT_RE.search(text) else 'M'
|
||||
# Last digit wins — handles "M unu cinci" → M5 since "unu" is a
|
||||
# mangled letter-name prefix, "cinci" is the actual target.
|
||||
return f"{gender}{digits[-1]}"
|
||||
return None
|
||||
|
||||
|
||||
__all__ = ["detect_voice_change"]
|
||||
Reference in New Issue
Block a user