fix: capture all intermediate text blocks from Claude tool-use responses

Switch from --output-format json to --output-format stream-json --verbose
so that _run_claude() parses all assistant text blocks (not just the final
result field). Discord/Telegram/WhatsApp now receive every intermediate
message Claude writes between tool calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CLAUDE.md

@@ -1,76 +1,79 @@
-# CLAUDE.md
+# Echo Core
 
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+**Tu ești Echo** — asistent personal AI al lui Marius. Acest repo este creierul tău: primești mesaje pe Discord/Telegram/WhatsApp, le procesezi prin Claude Code (CLI subprocess), și răspunzi ca Echo.
+
+Nu ești un tool de cod. Ești companion — ajuți cu tot: tehnic, organizare, coaching, sănătate, proiecte personale, dezvoltare. Cine ești și cum te comporți e definit în `personality/*.md`. **Respectă aceste fișiere întotdeauna.**
+
+## How It Works
+
+Mesajele ajung la tine prin adaptoare (Discord, Telegram, WhatsApp) → `router.py` → `claude_session.py` → Claude CLI subprocess → răspuns trimis înapoi.
+
+Personalitatea ta se construiește din `personality/*.md`, concatenate în ordine:
+- `IDENTITY.md` — cine ești
+- `SOUL.md` — principii, ton, granițe
+- `USER.md` — despre Marius
+- `AGENTS.md` — reguli operaționale, model selection, securitate
+- `HEARTBEAT.md` — verificări periodice
+- `TOOLS.md` — unelte disponibile
 
 ## Commands
 
 ```bash
-# Run all tests
+# Tests
 source .venv/bin/activate && pytest tests/
-
-# Run a single test file
-pytest tests/test_router.py
-
-# Run a specific test
 pytest tests/test_router.py::test_clear_command -v
 
-# Start the bot (via systemd)
+# Start
-systemctl --user start echo-core
+systemctl --user start echo-core          # systemd
-
+source .venv/bin/activate && python3 src/main.py  # manual
-# Start manually (foreground)
-source .venv/bin/activate && python3 src/main.py
 
 # WhatsApp bridge
 systemctl --user start echo-whatsapp-bridge
 
-# CLI diagnostics (eco = symlink to cli.py, installed by setup.sh)
+# CLI
 eco status
 eco doctor
 
-# Install dependencies
+# Dependencies
 source .venv/bin/activate && pip install -r requirements.txt
 ```
 
 ## Architecture
 
-**Message flow:** Adapter → `router.py` → `claude_session.py` → Claude CLI subprocess → response split → Adapter sends reply.
+**Flow:** Adapter → `router.py` → `claude_session.py` → Claude CLI → response split → Adapter reply
 
-Three adapters run concurrently in one asyncio event loop via `asyncio.gather()` in `src/main.py`:
+**Adapters** (concurrent, `asyncio.gather()` in `src/main.py`):
+- **Discord** (`src/adapters/discord_bot.py`) — slash commands, 2000 char split
+- **Telegram** (`src/adapters/telegram_bot.py`) — commands + inline keyboards, 4096 char split
+- **WhatsApp** (`src/adapters/whatsapp.py`) — polls Baileys bridge at `http://127.0.0.1:8098`, 4096 char split
 
-- **Discord** (`src/adapters/discord_bot.py`): discord.py slash commands, 2000 char split
+**Sessions** (`src/claude_session.py`): One persistent session per channel. `claude --resume <session_id>`. External messages wrapped in `[EXTERNAL CONTENT]` injection markers.
-- **Telegram** (`src/adapters/telegram_bot.py`): python-telegram-bot commands + inline keyboards, 4096 char split
-- **WhatsApp** (`src/adapters/whatsapp.py`): polls Node.js Baileys bridge (`bridge/whatsapp/index.js`) at `http://127.0.0.1:8098` every 2s, 4096 char split
 
-All adapters follow the same pattern: module-level `_config`, authorization helpers (`is_owner`, `is_admin`, `is_registered_channel`), `split_message()`, and routing through `route_message(channel_id, user_id, text, model)`.
+**State:** `sessions/active.json` — channel ID → `{session_id, model, message_count, ...}`
 
-**Claude sessions** (`src/claude_session.py`): Each channel has one persistent session. `send_message()` auto-starts or resumes via `claude --resume <session_id>`. System prompt is built by concatenating `personality/*.md` files. External messages are wrapped in `[EXTERNAL CONTENT]` injection markers. Sensitive env vars are stripped before subprocess execution.
+**Credentials** (`src/credential_store.py`): System keyring, service `"echo-core"`. Never secrets as CLI args.
 
-**Session state** lives in `sessions/active.json` — maps channel IDs to `{session_id, model, message_count, ...}`.
+**Config** (`src/config.py`): `config.json` with dot-notation. Namespaces: `channels`, `telegram_channels`, `whatsapp_channels`.
 
-**Credentials** (`src/credential_store.py`): All tokens stored in system keyring under service `"echo-core"`. Required: `discord_token`. Optional: `telegram_token`. Never pass secrets as CLI arguments — use stdin.
+**Scheduler** (`src/scheduler.py`): APScheduler + `cron/jobs.json`, isolated sessions.
 
-**Config** (`src/config.py`): Loads `config.json` with dot-notation access (`config.get("bot.name")`). Channel namespaces: `channels` (Discord), `telegram_channels`, `whatsapp_channels`.
+**Heartbeat** (`src/heartbeat.py`): Email, calendar, KB, git checks. Quiet hours 23-08.
 
-**Scheduler** (`src/scheduler.py`): APScheduler loading jobs from `cron/jobs.json`, runs Claude prompts in isolated sessions.
+**Memory** (`src/memory_search.py`): Ollama all-minilm embeddings (384 dim) + SQLite cosine similarity.
-
-**Heartbeat** (`src/heartbeat.py`): Periodic checks (email, calendar, KB, git), quiet hours 23-08.
-
-**Memory search** (`src/memory_search.py`): Ollama all-minilm embeddings (384 dim) + SQLite cosine similarity.
 
 ## Import Convention
 
-All modules use absolute imports from project root via `sys.path.insert(0, PROJECT_ROOT)`. Import as `from src.config import ...`, `from src.adapters.discord_bot import ...`. No circular imports — adapters → router → claude_session.
+Absolute imports via `sys.path.insert(0, PROJECT_ROOT)`: `from src.config import ...`, `from src.adapters.discord_bot import ...`. No circular imports.
 
 ## Key Files
 
 | Path | Role |
 |------|------|
-| `src/main.py` | Entry point — starts all adapters + scheduler + heartbeat |
+| `src/main.py` | Entry point — adapters + scheduler + heartbeat |
-| `src/router.py` | Dispatches commands vs Claude messages |
+| `src/router.py` | Commands vs Claude messages |
 | `src/claude_session.py` | Claude CLI wrapper with `--resume` |
-| `src/credential_store.py` | Keyring secrets (service: `echo-core`) |
+| `src/credential_store.py` | Keyring secrets |
-| `cli.py` | CLI tool (status, doctor, logs, secrets, cron, whatsapp) |
+| `cli.py` | CLI diagnostics (eco) |
-| `config.json` | Runtime config (channels, admins, models, bridges) |
+| `config.json` | Runtime config |
-| `setup.sh` | Interactive 10-step onboarding wizard |
+| `bridge/whatsapp/index.js` | Baileys + Express bridge, port 8098 |
-| `bridge/whatsapp/index.js` | Node.js Baileys + Express bridge on port 8098 |
+| `personality/*.md` | System prompt — cine ești |
-| `personality/*.md` | System prompt files concatenated in order |

src/claude_session.py

@@ -44,6 +44,55 @@ PERSONALITY_FILES = [
     "HEARTBEAT.md",
 ]
 
+# Tools allowed in non-interactive (-p) mode.
+# NOTE: curl/wget intentionally excluded (data exfiltration risk).
+# Use WebFetch/WebSearch for safe, read-only web access.
+# SSH/SCP/rsync restricted to local network (10.0.20.*).
+ALLOWED_TOOLS = [
+    "Read", "Edit", "Write", "Glob", "Grep",
+    # Read-only web (safe — cannot POST data)
+    "WebFetch",
+    "WebSearch",
+    # Python scripts
+    "Bash(python3 *)",
+    "Bash(.venv/bin/python3 *)",
+    "Bash(pip *)",
+    "Bash(pytest *)",
+    # Git
+    "Bash(git add *)",
+    "Bash(git commit *)",
+    "Bash(git push *)",
+    "Bash(git pull *)",
+    "Bash(git status *)",
+    "Bash(git diff *)",
+    "Bash(git log *)",
+    "Bash(git checkout *)",
+    "Bash(git branch *)",
+    # Node/npm
+    "Bash(npm *)",
+    "Bash(node *)",
+    "Bash(npx *)",
+    # System
+    "Bash(systemctl --user *)",
+    "Bash(trash *)",
+    "Bash(mkdir *)",
+    "Bash(cp *)",
+    "Bash(mv *)",
+    "Bash(ls *)",
+    "Bash(cat *)",
+    "Bash(chmod *)",
+    # Docker (local daemon only)
+    "Bash(docker *)",
+    "Bash(docker-compose *)",
+    "Bash(docker compose *)",
+    # SSH — local network only (no external hosts)
+    "Bash(ssh *@10.0.20.*)",
+    "Bash(ssh root@10.0.20.*)",
+    "Bash(ssh echo@10.0.20.*)",
+    "Bash(scp *10.0.20.*)",
+    "Bash(rsync *10.0.20.*)",
+]
+
 # Environment variables to REMOVE from Claude subprocess
 # (secrets, tokens, and vars that cause nested-session errors)
 _ENV_STRIP = {
@@ -108,7 +157,12 @@ def _save_sessions(data: dict) -> None:
 
 
 def _run_claude(cmd: list[str], timeout: int) -> dict:
-    """Run a Claude CLI command and return the parsed JSON output."""
+    """Run a Claude CLI command and return parsed output.
+
+    Expects ``--output-format stream-json --verbose``.  Parses the newline-
+    delimited JSON stream, collecting every text block from ``assistant``
+    messages and metadata from the final ``result`` line.
+    """
     if not shutil.which(CLAUDE_BIN):
         raise FileNotFoundError(
             "Claude CLI not found. "
@@ -135,12 +189,50 @@ def _run_claude(cmd: list[str], timeout: int) -> dict:
             f"Claude CLI error (exit {proc.returncode}): {detail}"
         )
 
-    try:
+    # --- Parse stream-json output ---
-        data = json.loads(proc.stdout)
+    text_blocks: list[str] = []
-    except json.JSONDecodeError as exc:
+    result_obj: dict | None = None
-        raise RuntimeError(f"Failed to parse Claude CLI output: {exc}")
 
-    return data
+    for line in proc.stdout.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            obj = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+
+        msg_type = obj.get("type")
+
+        if msg_type == "assistant":
+            # Extract text from content blocks
+            message = obj.get("message", {})
+            for block in message.get("content", []):
+                if block.get("type") == "text":
+                    text = block.get("text", "").strip()
+                    if text:
+                        text_blocks.append(text)
+
+        elif msg_type == "result":
+            result_obj = obj
+
+    if result_obj is None:
+        raise RuntimeError(
+            "Failed to parse Claude CLI output: no result line in stream"
+        )
+
+    # Build a dict compatible with the old json output format
+    combined_text = "\n\n".join(text_blocks) if text_blocks else result_obj.get("result", "")
+
+    return {
+        "result": combined_text,
+        "session_id": result_obj.get("session_id", ""),
+        "usage": result_obj.get("usage", {}),
+        "total_cost_usd": result_obj.get("total_cost_usd", 0),
+        "cost_usd": result_obj.get("cost_usd", 0),
+        "duration_ms": result_obj.get("duration_ms", 0),
+        "num_turns": result_obj.get("num_turns", 0),
+    }
 
 
 # ---------------------------------------------------------------------------
@@ -199,8 +291,9 @@ def start_session(
     cmd = [
         CLAUDE_BIN, "-p", wrapped_message,
         "--model", model,
-        "--output-format", "json",
+        "--output-format", "stream-json", "--verbose",
         "--system-prompt", system_prompt,
+        "--allowedTools", *ALLOWED_TOOLS,
     ]
 
     _t0 = time.monotonic()
@@ -208,7 +301,7 @@ def start_session(
     _elapsed_ms = int((time.monotonic() - _t0) * 1000)
 
     for field in ("result", "session_id"):
-        if field not in data:
+        if not data.get(field):
             raise RuntimeError(
                 f"Claude CLI response missing required field: {field}"
             )
@@ -267,14 +360,15 @@ def resume_session(
     cmd = [
         CLAUDE_BIN, "-p", wrapped_message,
         "--resume", session_id,
-        "--output-format", "json",
+        "--output-format", "stream-json", "--verbose",
+        "--allowedTools", *ALLOWED_TOOLS,
     ]
 
     _t0 = time.monotonic()
     data = _run_claude(cmd, timeout)
     _elapsed_ms = int((time.monotonic() - _t0) * 1000)
 
-    if "result" not in data:
+    if not data.get("result"):
         raise RuntimeError(
             "Claude CLI response missing required field: result"
         )

tests/test_claude_session.py

@@ -28,21 +28,45 @@ from src.claude_session import (
 # Helpers
 # ---------------------------------------------------------------------------
 
-FAKE_CLI_RESPONSE = {
+FAKE_RESULT_LINE = {
     "type": "result",
     "subtype": "success",
     "session_id": "sess-abc-123",
     "result": "Hello from Claude!",
     "cost_usd": 0.004,
+    "total_cost_usd": 0.004,
     "duration_ms": 1500,
     "num_turns": 1,
+    "usage": {"input_tokens": 100, "output_tokens": 50},
+}
+
+FAKE_ASSISTANT_LINE = {
+    "type": "assistant",
+    "message": {
+        "content": [{"type": "text", "text": "Hello from Claude!"}],
+    },
 }
 
 
+def _make_stream(*assistant_texts, result_override=None):
+    """Build stream-json stdout with assistant + result lines."""
+    lines = []
+    for text in assistant_texts:
+        lines.append(json.dumps({
+            "type": "assistant",
+            "message": {"content": [{"type": "text", "text": text}]},
+        }))
+    result = dict(FAKE_RESULT_LINE)
+    if result_override:
+        result.update(result_override)
+    lines.append(json.dumps(result))
+    return "\n".join(lines)
+
+
 def _make_proc(stdout=None, returncode=0, stderr=""):
-    """Build a fake subprocess.CompletedProcess."""
+    """Build a fake subprocess.CompletedProcess with stream-json output."""
     if stdout is None:
-        stdout = json.dumps(FAKE_CLI_RESPONSE)
+        stdout = _make_stream("Hello from Claude!")
     proc = MagicMock(spec=subprocess.CompletedProcess)
     proc.stdout = stdout
     proc.stderr = stderr
@@ -153,10 +177,20 @@ class TestSafeEnv:
 class TestRunClaude:
     @patch("shutil.which", return_value="/usr/bin/claude")
     @patch("subprocess.run")
-    def test_returns_parsed_json(self, mock_run, mock_which):
+    def test_returns_parsed_stream(self, mock_run, mock_which):
         mock_run.return_value = _make_proc()
         result = _run_claude(["claude", "-p", "hi"], timeout=30)
-        assert result == FAKE_CLI_RESPONSE
+        assert result["result"] == "Hello from Claude!"
+        assert result["session_id"] == "sess-abc-123"
+        assert "usage" in result
+
+    @patch("shutil.which", return_value="/usr/bin/claude")
+    @patch("subprocess.run")
+    def test_collects_multiple_text_blocks(self, mock_run, mock_which):
+        stdout = _make_stream("First message", "Second message", "Third message")
+        mock_run.return_value = _make_proc(stdout=stdout)
+        result = _run_claude(["claude", "-p", "hi"], timeout=30)
+        assert result["result"] == "First message\n\nSecond message\n\nThird message"
 
     @patch("shutil.which", return_value="/usr/bin/claude")
     @patch("subprocess.run")
@@ -176,9 +210,11 @@ class TestRunClaude:
 
     @patch("shutil.which", return_value="/usr/bin/claude")
     @patch("subprocess.run")
-    def test_invalid_json_raises(self, mock_run, mock_which):
+    def test_no_result_line_raises(self, mock_run, mock_which):
-        mock_run.return_value = _make_proc(stdout="not json {{{")
+        # Stream with only an assistant line but no result line
-        with pytest.raises(RuntimeError, match="Failed to parse"):
+        stdout = json.dumps({"type": "assistant", "message": {"content": []}})
+        mock_run.return_value = _make_proc(stdout=stdout)
+        with pytest.raises(RuntimeError, match="no result line"):
             _run_claude(["claude", "-p", "hi"], timeout=30)
 
     @patch("shutil.which", return_value=None)
@@ -299,7 +335,7 @@ class TestStartSession:
 
     @patch("shutil.which", return_value="/usr/bin/claude")
     @patch("subprocess.run")
-    def test_missing_result_field_raises(
+    def test_missing_result_line_raises(
         self, mock_run, mock_which, tmp_path, monkeypatch
     ):
         sessions_dir = tmp_path / "sessions"
@@ -308,15 +344,16 @@ class TestStartSession:
         monkeypatch.setattr(
             claude_session, "_SESSIONS_FILE", sessions_dir / "active.json"
         )
-        bad_response = {"session_id": "abc"}  # missing "result"
+        # Stream with no result line at all
-        mock_run.return_value = _make_proc(stdout=json.dumps(bad_response))
+        bad_stream = json.dumps({"type": "assistant", "message": {"content": []}})
+        mock_run.return_value = _make_proc(stdout=bad_stream)
 
-        with pytest.raises(RuntimeError, match="missing required field"):
+        with pytest.raises(RuntimeError, match="no result line"):
             start_session("general", "Hello")
 
     @patch("shutil.which", return_value="/usr/bin/claude")
     @patch("subprocess.run")
-    def test_missing_session_id_field_raises(
+    def test_missing_session_id_gives_empty_string(
         self, mock_run, mock_which, tmp_path, monkeypatch
     ):
         sessions_dir = tmp_path / "sessions"
@@ -325,8 +362,10 @@ class TestStartSession:
         monkeypatch.setattr(
             claude_session, "_SESSIONS_FILE", sessions_dir / "active.json"
         )
-        bad_response = {"result": "hello"}  # missing "session_id"
+        # Result line without session_id → _run_claude returns "" for session_id
-        mock_run.return_value = _make_proc(stdout=json.dumps(bad_response))
+        # → start_session checks for empty session_id
+        bad_stream = _make_stream("hello", result_override={"session_id": None})
+        mock_run.return_value = _make_proc(stdout=bad_stream)
 
         with pytest.raises(RuntimeError, match="missing required field"):
             start_session("general", "Hello")