feat(voice): DAVE E2E + full voice UX (squash of voice/dave-recv)
Squashed branch: voice/dave-recv → master. Closes Pas 12 (DAVE E2E) and lands voice-mode UX polish + verbal voice control on top of the Pas 1-10 scaffolding already on master. ## DAVE E2E receive-side decrypt (e4f3177) Vendored fork: discord-ext-voice-recv 0.5.3a+echo.dave1. Patches the receive pipeline to handle Discord's mandatory DAVE encryption on voice gateway v=8. - `_maybe_dave_decrypt`: uses davey.can_passthrough(user_id) as primary gate, falls through to dave.decrypt for DAVE-epoch peers, drops on decrypt failure without killing the reader thread. - VAD fix: silero-vad v5+ requires exactly 512 samples; our 100ms window (1600 samples) was silently raising ValueError → STT never fired. Now slice into 512-sample chunks. - Whisper: bumped beam_size 1→5 and added RO initial_prompt. - Tests: 11 DAVE unit tests + 2 callback integration tests + contract test with fork-version guard. ## Voice UX polish (d1bc77e) - Killed the 3s "mă gândesc" filler (always collided with Claude p50 4-7s). - Barge-in via `ttsq.clear()` at top of `on_segment_done`. - DTX silence-flush poller (200ms tick) — Discord stops sending RTP packets when silent, so the inline silence-check in sink.write() never fired for trailing audio; background thread handles it. - `EchoStreamingAudioSource.read()` non-blocking — old `get_frame(timeout=0.1)` wrecked Discord's 20ms cadence and the client interpreted bursts as stuttering (Marius heard "4 de minute" instead of full sentence). - RO time expansion: 23:09 → "douăzeci și trei și nouă minute". - Supertonic Unicode sanitize centralized in tools/tts.py. - Whisper local_files_only=True — no HF metadata GET on each startup. - Diagnostic logging through sink → VAD → Claude stream → TTS chain. ## Voice mode iteration (e589e48) - `personality/VOICE_MODE.md` — voice-tailored system prompt (short, no markdown, no abbreviations, time without seconds, distances in "mii"/"milioane"); plumbed via build_system_prompt(voice_mode=True). - Isolated voice session key `voice:<channel_id>` — voice doesn't share context with text adapter on the same channel; auto-applied without /clear ceremony. /clear drops both keys. - Metric units + Romanian thousands (normalize.py): "384.000 km" → "trei sute optzeci și patru de mii de kilometri" with feminine-correct pluralization and "de" particle for ≥20. - `/voice setvoice <M1-F5>` slash command with native autocomplete; swaps live + persists voice.default_voice to config.json. - Verbal voice change (src/voice/voice_commands.py + 29 tests) — "schimbă vocea pe M5", "voce em cinci", with permissive substring fallback for Whisper-mangled forms like "Mâcinci"=M5 and "unul cinci"=M5. Whisper initial_prompt now lists voice vocabulary to bias STT toward clean outputs. - Fast barge-in: VAD ≥2 consecutive windows (~200ms) on Marius's user while Echo has pending TTS frames → cut him off mid-sentence so user doesn't wait the full silence + STT cycle. Acoustic echo bleed-through still requires headphones (no AEC). ## Test suite 130 voice + router tests pass (test_voice_recv_dave, test_voice_session_cleanup, test_voice_adapter_contract, test_voice_normalize, test_voice_commands, test_router). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
60
vendor/discord-ext-voice-recv/VENDOR_INFO.md
vendored
60
vendor/discord-ext-voice-recv/VENDOR_INFO.md
vendored
@@ -1,22 +1,76 @@
|
||||
# Vendored: discord-ext-voice-recv
|
||||
|
||||
**Upstream:** https://github.com/imayhaveborkedit/discord-ext-voice-recv
|
||||
**Pinned commit:** `ac04ea7b0941112e83767cf1c1469b408fa06748` (bump version 0.5.3a)
|
||||
**Pinned commit:** `ac04ea7b0941112e83767cf1c1469b408fa06748` (bump version 0.5.3a, master HEAD Jun 2025)
|
||||
**Vendored at:** 2026-05-27
|
||||
**Echo Core fork version:** `0.5.3a+echo.dave1` (PEP 440 local segment)
|
||||
**Reason:** Discord voice protocol is fragile, upstream is hobby fork. Adapter
|
||||
layer in `src/voice/_discord_voice_adapter.py` isolates upstream churn — if this
|
||||
package breaks, swap to py-cord by rewriting only that file.
|
||||
|
||||
## Update procedure
|
||||
## Echo Core patch: `+echo.dave1` (DAVE E2E receive-side decrypt)
|
||||
|
||||
### Why
|
||||
|
||||
Discord enforces DAVE (E2E media encryption) on voice gateway `v=8` whenever the
|
||||
bot advertises `max_dave_protocol_version > 0` in IDENTIFY. discord.py 2.7.1 (the
|
||||
version Echo Core pins) does so unconditionally — Discord then closes the WS
|
||||
with code **4017** if the bot opts out by sending `max_dave_protocol_version=0`.
|
||||
DAVE is **mandatory**.
|
||||
|
||||
Audio received from a DAVE-active room is **dual-wrapped**: transport layer
|
||||
(`aead_xchacha20_poly1305_rtpsize`) + DAVE E2E. Upstream voice-recv decrypts
|
||||
only the transport layer, then hands DAVE ciphertext to libopus, which raises
|
||||
`OpusError: corrupted stream` on every packet.
|
||||
|
||||
### Patch shape
|
||||
|
||||
~30 lines, all in `discord/ext/voice_recv/reader.py`:
|
||||
|
||||
1. Module-level optional `davey` import (no-op when missing).
|
||||
2. `AudioReader._maybe_dave_decrypt(rtp_packet) -> Optional[bytes]` — gate logic
|
||||
mirrors discord.py 2.7.1 send-side `can_encrypt` exactly. Returns the
|
||||
DAVE-unwrapped payload, the original payload (DAVE inactive), or `None` to
|
||||
drop the packet (unknown SSRC, decrypt failure).
|
||||
3. 4-line hook in `callback()` between transport-decrypt and `feed_rtp`:
|
||||
overwrites `rtp_packet.decrypted_data` in place, or returns early to drop.
|
||||
|
||||
The post-decrypt `is_silence()` check (formerly at reader.py:172) still works
|
||||
because we overwrite `decrypted_data` in place — silence frames produced by
|
||||
davey reach the existing check unchanged.
|
||||
|
||||
### Dependency
|
||||
|
||||
`davey==0.1.5` — matches discord.py 2.7.1 expectation. Pin in
|
||||
`echo-core/requirements.txt`. The import is optional at module level so tests
|
||||
and non-DAVE environments still run; the gate degrades to a bypass.
|
||||
|
||||
### Re-sync strategy
|
||||
|
||||
When upstream voice-recv adds DAVE support natively:
|
||||
|
||||
1. Drop the three patch hunks in `reader.py` (davey import block,
|
||||
`_maybe_dave_decrypt` method, hook in `callback()`).
|
||||
2. Revert `__version__` to upstream value in `__init__.py`.
|
||||
3. Update `Pinned commit` below.
|
||||
4. Run `pytest tests/test_voice_recv_dave.py tests/test_voice_adapter_contract.py`.
|
||||
|
||||
The contract test `test_voice_recv_fork_version` asserts `__version__ ==
|
||||
'0.5.3a+echo.dave1'` and will fail fast on any accidental wipe during a careless
|
||||
upstream sync — forcing a conscious decision to either re-port or drop the
|
||||
patch.
|
||||
|
||||
## Update procedure (vanilla upstream sync)
|
||||
|
||||
```bash
|
||||
cd vendor/discord-ext-voice-recv
|
||||
git fetch origin master
|
||||
git log HEAD..origin/master --oneline # review what changed
|
||||
git checkout <new-commit>
|
||||
# RE-APPLY the +echo.dave1 patch if upstream still lacks DAVE
|
||||
cd ../..
|
||||
source .venv/bin/activate && pip install -e vendor/discord-ext-voice-recv --force-reinstall
|
||||
pytest tests/test_voice_adapter_contract.py -v # MUST PASS — contract guard
|
||||
pytest tests/test_voice_adapter_contract.py tests/test_voice_recv_dave.py -v # MUST PASS — contract + DAVE guards
|
||||
```
|
||||
|
||||
Update this file's `Pinned commit` after a successful upgrade.
|
||||
|
||||
@@ -17,4 +17,4 @@ __title__ = 'discord.ext.voice_recv'
|
||||
__author__ = 'Imayhaveborkedit'
|
||||
__license__ = 'MIT'
|
||||
__copyright__ = 'Copyright 2021-present Imayhaveborkedit'
|
||||
__version__ = '0.5.3a'
|
||||
__version__ = '0.5.3a+echo.dave1'
|
||||
|
||||
@@ -19,6 +19,15 @@ try:
|
||||
except ImportError as e:
|
||||
raise RuntimeError("pynacl is required") from e
|
||||
|
||||
# Echo Core +echo.dave1 patch: DAVE E2E receive-side decrypt. See VENDOR_INFO.md.
|
||||
try:
|
||||
import davey
|
||||
_MEDIA_TYPE_AUDIO = davey.MediaType.audio
|
||||
_HAS_DAVE = True
|
||||
except ImportError:
|
||||
_MEDIA_TYPE_AUDIO = None
|
||||
_HAS_DAVE = False
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from typing import Optional, Callable, Any, Dict, Literal, Union
|
||||
|
||||
@@ -133,12 +142,63 @@ class AudioReader:
|
||||
def _is_ip_discovery_packet(self, data: bytes) -> bool:
|
||||
return len(data) == 74 and data[1] == 0x02
|
||||
|
||||
def _maybe_dave_decrypt(self, rtp_packet) -> Optional[bytes]:
|
||||
"""DAVE E2E layer applied after transport decrypt.
|
||||
|
||||
Returns the (possibly DAVE-unwrapped) opus payload, or None to drop the
|
||||
packet. No-op when DAVE is inactive — non-DAVE rooms and environments
|
||||
without `davey` installed pass through unchanged.
|
||||
|
||||
NOTE: `is_silence()` is NOT checked here. In a DAVE-active room the
|
||||
transport-decrypted payload is ciphertext, so `is_silence()` (which
|
||||
compares to plaintext OPUS_SILENCE ``b'\\xf8\\xff\\xfe'``) never matches.
|
||||
Silence frames are handled either by davey.decrypt returning plaintext
|
||||
silence (then caught at the existing post-decrypt silence check on
|
||||
``decrypted_data``), or dropped via the decrypt-raises path. The
|
||||
existing post-decrypt silence check continues to work because we
|
||||
overwrite ``decrypted_data`` in place.
|
||||
"""
|
||||
if not _HAS_DAVE:
|
||||
return rtp_packet.decrypted_data
|
||||
conn = self.voice_client._connection
|
||||
if getattr(conn, 'dave_protocol_version', 0) == 0:
|
||||
return rtp_packet.decrypted_data
|
||||
dave = getattr(conn, 'dave_session', None)
|
||||
if dave is None or not dave.ready:
|
||||
return rtp_packet.decrypted_data
|
||||
user_id = self.voice_client._ssrc_to_id.get(rtp_packet.ssrc)
|
||||
if user_id is None:
|
||||
# ACCEPTED REGRESSION: davey requires per-user key. When SPEAKING
|
||||
# event races behind the first audio packet, we drop 1-5 packets
|
||||
# (~40-200ms) per new speaker per session.
|
||||
return None
|
||||
# can_passthrough(user_id) mirrors Discord's protocol: when this user's
|
||||
# decryptor is in passthrough mode, packets are not DAVE-wrapped and
|
||||
# must be returned as-is. Otherwise davey.decrypt unwraps DAVE E2E.
|
||||
try:
|
||||
if dave.can_passthrough(user_id):
|
||||
return rtp_packet.decrypted_data
|
||||
except Exception as e:
|
||||
log.debug("can_passthrough check failed for ssrc=%s user=%s: %s: %s",
|
||||
rtp_packet.ssrc, user_id, type(e).__name__, e)
|
||||
try:
|
||||
return dave.decrypt(user_id, _MEDIA_TYPE_AUDIO, rtp_packet.decrypted_data)
|
||||
except Exception as e:
|
||||
log.debug("DAVE decrypt failed for ssrc=%s user=%s: %s: %s",
|
||||
rtp_packet.ssrc, user_id, type(e).__name__, e)
|
||||
return None
|
||||
|
||||
def callback(self, packet_data: bytes) -> None:
|
||||
packet = rtp_packet = rtcp_packet = None
|
||||
try:
|
||||
if not rtp.is_rtcp(packet_data):
|
||||
packet = rtp_packet = rtp.decode_rtp(packet_data)
|
||||
packet.decrypted_data = self.decryptor.decrypt_rtp(packet)
|
||||
# Echo Core +echo.dave1: DAVE E2E layer (no-op when inactive).
|
||||
dave_payload = self._maybe_dave_decrypt(rtp_packet)
|
||||
if dave_payload is None:
|
||||
return # drop packet, do not feed_rtp; reader thread stays alive
|
||||
rtp_packet.decrypted_data = dave_payload
|
||||
else:
|
||||
packet = rtcp_packet = rtp.decode_rtcp(self.decryptor.decrypt_rtcp(packet_data))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user