Implements the approved plan to replace the broken regex/index-master
extraction with an LLM-subagent pipeline. Four parallel lanes:
Lane A — scripts/extract_common.py (PDF/docx/doc/pptx/html/zip, no
max_pages truncation), normalize_sources.py, chunk_sources.py
(~20pg chunks + overlap, manifest registry), activity_schema.json.
Lane B — app/config_taxonomy.py (16 fixed category slugs), schema
rebuilt from scratch in app/models/ with content_type, language,
source_files, source_excerpt, normalized_name, extraction_confidence,
needs_review; FTS5 + 3 triggers extended with materials_list and
skills_developed.
Lane C — build_database.py (--rebuild, atomic swap, schema + fuzzy
source_excerpt validation, dedup with needs_review band),
validate_extractions.py, review_queue.py, new run_extraction.py
orchestrator, SUBAGENT_PROMPT.md.
Lane D — search.py content_type/language filters (default search
excludes non-game content), E7 schema-compat audit; fixed a NULL
keywords AttributeError in _boost_search_relevance.
Removes 8 orphaned/dead scripts and app/services/parser.py +
indexer.py. Adds tests/ (70 passing, 1 skipped — libreoffice absent).
Note: Lane D made one additive edit to app/models/database.py
(_update_category_counts) to surface content_type/language in
get_filter_options, outside its nominal lane boundary but after
Lane B completed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Major Changes:
- Migrated from prototype to production architecture
- Implemented modular Flask app with models/services/web layers
- Added Docker containerization with docker-compose
- Switched to Pipenv for dependency management
- Built advanced parser extracting 63 real activities from INDEX_MASTER
- Implemented SQLite FTS5 full-text search
- Created minimalist, responsive web interface
- Added comprehensive documentation and deployment guides
Technical Improvements:
- Clean separation of concerns (models, services, web)
- Enhanced database schema with FTS5 indexing
- Dynamic filters populated from real data
- Production-ready configuration management
- Security best practices implementation
- Health monitoring and API endpoints
Removed Legacy Files:
- Old src/ directory structure
- Static requirements.txt (replaced by Pipfile)
- Test and debug files
- Temporary cache files
Current Status:
- 63 activities indexed across 8 categories
- Full-text search operational
- Docker deployment ready
- Production documentation complete
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
