feat(voice): DAVE E2E + full voice UX (squash of voice/dave-recv)
Squashed branch: voice/dave-recv → master. Closes Pas 12 (DAVE E2E) and lands voice-mode UX polish + verbal voice control on top of the Pas 1-10 scaffolding already on master. ## DAVE E2E receive-side decrypt (e4f3177) Vendored fork: discord-ext-voice-recv 0.5.3a+echo.dave1. Patches the receive pipeline to handle Discord's mandatory DAVE encryption on voice gateway v=8. - `_maybe_dave_decrypt`: uses davey.can_passthrough(user_id) as primary gate, falls through to dave.decrypt for DAVE-epoch peers, drops on decrypt failure without killing the reader thread. - VAD fix: silero-vad v5+ requires exactly 512 samples; our 100ms window (1600 samples) was silently raising ValueError → STT never fired. Now slice into 512-sample chunks. - Whisper: bumped beam_size 1→5 and added RO initial_prompt. - Tests: 11 DAVE unit tests + 2 callback integration tests + contract test with fork-version guard. ## Voice UX polish (d1bc77e) - Killed the 3s "mă gândesc" filler (always collided with Claude p50 4-7s). - Barge-in via `ttsq.clear()` at top of `on_segment_done`. - DTX silence-flush poller (200ms tick) — Discord stops sending RTP packets when silent, so the inline silence-check in sink.write() never fired for trailing audio; background thread handles it. - `EchoStreamingAudioSource.read()` non-blocking — old `get_frame(timeout=0.1)` wrecked Discord's 20ms cadence and the client interpreted bursts as stuttering (Marius heard "4 de minute" instead of full sentence). - RO time expansion: 23:09 → "douăzeci și trei și nouă minute". - Supertonic Unicode sanitize centralized in tools/tts.py. - Whisper local_files_only=True — no HF metadata GET on each startup. - Diagnostic logging through sink → VAD → Claude stream → TTS chain. ## Voice mode iteration (e589e48) - `personality/VOICE_MODE.md` — voice-tailored system prompt (short, no markdown, no abbreviations, time without seconds, distances in "mii"/"milioane"); plumbed via build_system_prompt(voice_mode=True). - Isolated voice session key `voice:<channel_id>` — voice doesn't share context with text adapter on the same channel; auto-applied without /clear ceremony. /clear drops both keys. - Metric units + Romanian thousands (normalize.py): "384.000 km" → "trei sute optzeci și patru de mii de kilometri" with feminine-correct pluralization and "de" particle for ≥20. - `/voice setvoice <M1-F5>` slash command with native autocomplete; swaps live + persists voice.default_voice to config.json. - Verbal voice change (src/voice/voice_commands.py + 29 tests) — "schimbă vocea pe M5", "voce em cinci", with permissive substring fallback for Whisper-mangled forms like "Mâcinci"=M5 and "unul cinci"=M5. Whisper initial_prompt now lists voice vocabulary to bias STT toward clean outputs. - Fast barge-in: VAD ≥2 consecutive windows (~200ms) on Marius's user while Echo has pending TTS frames → cut him off mid-sentence so user doesn't wait the full silence + STT cycle. Acoustic echo bleed-through still requires headphones (no AEC). ## Test suite 130 voice + router tests pass (test_voice_recv_dave, test_voice_session_cleanup, test_voice_adapter_contract, test_voice_normalize, test_voice_commands, test_router). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -21,6 +21,14 @@ from typing import Optional
|
||||
import discord
|
||||
from discord import app_commands
|
||||
|
||||
# Optional DAVE dep (mandatory at runtime when discord.py 2.7.1 is paired with
|
||||
# Discord voice gateway v=8; tolerated missing in tests / dev environments).
|
||||
try:
|
||||
import davey
|
||||
_HAS_DAVE = True
|
||||
except ImportError:
|
||||
_HAS_DAVE = False
|
||||
|
||||
from src.config import Config
|
||||
from src.voice.pipeline import (
|
||||
VoiceSession,
|
||||
@@ -28,7 +36,7 @@ from src.voice.pipeline import (
|
||||
_get_whisper_model,
|
||||
_get_silero_vad,
|
||||
)
|
||||
from src.voice.tts_stream import TTSQueue
|
||||
from src.voice.tts_stream import TTSQueue, EchoStreamingAudioSource
|
||||
from src.voice._discord_voice_adapter import connect_voice
|
||||
|
||||
log = logging.getLogger("echo-core.discord.voice")
|
||||
@@ -53,6 +61,11 @@ async def warmup_models() -> None:
|
||||
"""
|
||||
global _voice_load_error
|
||||
try:
|
||||
if not discord.opus.is_loaded():
|
||||
discord.opus.load_opus("libopus.so.0")
|
||||
if _HAS_DAVE:
|
||||
log.info("DAVE protocol v%d available (davey %s)",
|
||||
davey.DAVE_PROTOCOL_VERSION, davey.__version__)
|
||||
await asyncio.to_thread(_get_whisper_model)
|
||||
await asyncio.to_thread(_get_silero_vad)
|
||||
log.info("Voice models warm")
|
||||
@@ -167,11 +180,24 @@ def register(tree: app_commands.CommandTree, bot: discord.Client) -> app_command
|
||||
)
|
||||
return
|
||||
_voice_sessions[guild_id] = session
|
||||
# Wake-up beep
|
||||
# Start TTS streaming source for the entire session. Chain the
|
||||
# wake-up beep via `after=` so streaming takes over when beep ends.
|
||||
def _start_stream(error: Optional[Exception] = None) -> None:
|
||||
if error is not None:
|
||||
log.warning("Beep playback ended with error: %s", error)
|
||||
try:
|
||||
vc.play(EchoStreamingAudioSource(ttsq))
|
||||
log.info("TTS streaming source attached")
|
||||
except Exception:
|
||||
log.exception("EchoStreamingAudioSource attach failed")
|
||||
try:
|
||||
vc.play(discord.FFmpegPCMAudio("assets/voice/beep_200ms.wav"))
|
||||
vc.play(
|
||||
discord.FFmpegPCMAudio("assets/voice/beep_200ms.wav"),
|
||||
after=_start_stream,
|
||||
)
|
||||
except Exception:
|
||||
log.warning("Beep playback skipped", exc_info=True)
|
||||
log.warning("Beep playback skipped, starting stream directly", exc_info=True)
|
||||
_start_stream()
|
||||
# Attach sink
|
||||
try:
|
||||
bot_user_id = int(bot.user.id) if bot.user is not None else 0
|
||||
@@ -220,6 +246,45 @@ def register(tree: app_commands.CommandTree, bot: discord.Client) -> app_command
|
||||
log.warning("Presence reset skipped", exc_info=True)
|
||||
await interaction.followup.send("Plecat.", ephemeral=True)
|
||||
|
||||
_VOICE_CHOICES = [
|
||||
app_commands.Choice(name=v, value=v)
|
||||
for v in ("M1", "M2", "M3", "M4", "M5", "F1", "F2", "F3", "F4", "F5")
|
||||
]
|
||||
|
||||
@voice_group.command(name="setvoice", description="Schimbă vocea Echo (M1-M5 sau F1-F5)")
|
||||
@app_commands.describe(voice="Voce nouă")
|
||||
@app_commands.choices(voice=_VOICE_CHOICES)
|
||||
async def setvoice(
|
||||
interaction: discord.Interaction,
|
||||
voice: app_commands.Choice[str],
|
||||
) -> None:
|
||||
await interaction.response.defer(ephemeral=True)
|
||||
new_voice = voice.value
|
||||
# Live-swap on the active session if Echo is in voice on this guild.
|
||||
guild_id = interaction.guild.id if interaction.guild else None
|
||||
session = _voice_sessions.get(guild_id) if guild_id is not None else None
|
||||
live_swapped = False
|
||||
if session is not None and session.ttsq is not None:
|
||||
session.ttsq.voice_id = new_voice
|
||||
live_swapped = True
|
||||
# Persist as the new default for future sessions.
|
||||
try:
|
||||
cfg = Config()
|
||||
cfg.set("voice.default_voice", new_voice)
|
||||
cfg.save()
|
||||
except Exception as e:
|
||||
log.warning("config save failed for new default voice: %s", e)
|
||||
await interaction.followup.send(
|
||||
f"Voce schimbată live ({new_voice}), dar config-ul nu s-a salvat: {e}",
|
||||
ephemeral=True,
|
||||
)
|
||||
return
|
||||
if live_swapped:
|
||||
msg = f"Vocea schimbată **live** pe {new_voice}. Următoarea frază va folosi vocea nouă."
|
||||
else:
|
||||
msg = f"Default voce setată {new_voice}. Va intra în vigoare la următorul /voice join."
|
||||
await interaction.followup.send(msg, ephemeral=True)
|
||||
|
||||
@voice_group.command(name="doctor", description="Verifică voice stack")
|
||||
async def doctor(interaction: discord.Interaction) -> None:
|
||||
await interaction.response.defer(ephemeral=True)
|
||||
|
||||
Reference in New Issue
Block a user