romfast/vending_data_intelligence_report
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement Dashboard consolidation + Performance logging

Browse Source 
Features:
- Add unified "Dashboard Complet" sheet (Excel) with all 9 sections
- Add unified "Dashboard Complet" page (PDF) with key metrics
- Fix VALOARE_ANTERIOARA NULL bug (use sumar_executiv_yoy directly)
- Add PerformanceLogger class for timing analysis
- Remove redundant consolidated sheets (keep only Dashboard Complet)

Bug fixes:
- Fix Excel formula error (=== interpreted as formula, changed to >>>)
- Fix args.output → args.output_dir in perf.summary()

Performance analysis:
- Add PERFORMANCE_ANALYSIS.md with detailed breakdown
- SQL queries take 94% of runtime (31 min), Excel/PDF only 1%
- Identified slow queries for optimization

Documentation:
- Update CLAUDE.md with new structure
- Add context handover for query optimization task

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Marius Mutu
2025-12-11 13:33:02 +02:00
parent a2ad4c7ed2
commit 9e9ddec014
20 changed files with 2400 additions and 959 deletions
Download Patch File Download Diff File Expand all files Collapse all files

129
CLAUDE.md
View File

@@ -4,90 +4,107 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
Data Intelligence Report Generator for ERP ROA (Oracle Database). Generates Excel and PDF business intelligence reports with sales analytics, margin analysis, stock tracking, and alerts.
Data Intelligence Report Generator for ERP ROA (Oracle Database). Generates Excel and PDF business intelligence reports with sales analytics, margin analysis, stock tracking, financial indicators, and alerts.
## Commands
### Option 1: Virtual Environment (WSL or Windows)
```bash
# Create and activate virtual environment
# Virtual Environment setup
python -m venv .venv
source .venv/bin/activate # Linux/WSL
# or: .venv\Scripts\activate # Windows
# Install dependencies
pip install -r requirements.txt
# Run report
# Run report (default: last 12 months)
python main.py
```
### Option 2: Docker (Windows Docker Desktop / Linux)
```bash
# Copy and configure environment
cp .env.example .env
# Edit .env with your Oracle credentials
# Run with docker-compose
docker-compose run --rm report-generator
# Or with custom months
docker-compose run --rm report-generator python main.py --months 6
```
### Common Options
```bash
# Run with custom period
# Custom period
python main.py --months 6
# Custom output directory
python main.py --output-dir /path/to/output
# Docker alternative
docker-compose run --rm report-generator
```
## Oracle Connection from Different Environments
## Oracle Connection
| Environment | ORACLE_HOST value |
|-------------|-------------------|
| Windows native | `127.0.0.1` |
| WSL | Windows IP (run: `cat /etc/resolv.conf \| grep nameserver`) |
| Docker | `host.docker.internal` (automatic in docker-compose) |
| WSL | Windows IP (`cat /etc/resolv.conf \| grep nameserver`) |
| Docker | `host.docker.internal` |
## Architecture
**Entry point**: `main.py` - CLI interface, orchestrates query execution and report generation
```
main.py # Entry point, orchestrates everything
├── config.py # .env loader, thresholds (RECOMMENDATION_THRESHOLDS)
├── queries.py # SQL queries in QUERIES dict with metadata
├── recommendations.py # RecommendationsEngine - auto-generates alerts
└── report_generator.py # Excel/PDF generators
```
**Data flow**:
1. `config.py` loads Oracle connection settings from `.env` file
2. `queries.py` contains all SQL queries in a `QUERIES` dictionary with metadata (title, description, params)
3. `main.py` executes queries via `OracleConnection` context manager, stores results in `results` dict
4. `report_generator.py` receives dataframes and generates:
- `ExcelReportGenerator`: Multi-sheet workbook with conditional formatting
- `PDFReportGenerator`: Executive summary with charts via ReportLab
1. `main.py` executes queries via `OracleConnection` context manager
2. Results stored in `results` dict (query_name → DataFrame)
3. Consolidation logic merges related DataFrames (e.g., KPIs + YoY)
4. `ExcelReportGenerator` creates consolidated sheets + detail sheets
5. `PDFReportGenerator` creates consolidated pages + charts
**Key patterns**:
- Queries use parameterized `:months` for configurable analysis period
- Sheet order in `main.py:sheet_order` controls Excel tab sequence
- Charts are generated via matplotlib, converted to images for PDF
**Report structure** (after consolidation):
- **Excel**: 4 consolidated sheets (Vedere Ansamblu, Indicatori Venituri, Clienti si Risc, Tablou Financiar) + detail sheets
- **PDF**: Consolidated pages with multiple sections + charts + detail tables
## Oracle Database Schema
## Key Code Locations
Required views: `fact_vfacturi2`, `fact_vfacturi_detalii`, `vnom_articole`, `vnom_parteneri`, `vstoc`, `vrul`
Filter conventions:
- `sters = 0` excludes deleted records
- `tip NOT IN (7, 8, 9, 24)` excludes returns/credit notes
- Account codes: `341`, `345` = own production; `301` = raw materials
| What | Where |
|------|-------|
| SQL queries | `queries.py` - constants like `SUMAR_EXECUTIV`, `CONCENTRARE_RISC_YOY` |
| Query registry | `queries.py:QUERIES` dict |
| Sheet order | `main.py:sheet_order` list (~line 242) |
| Consolidated sheets | `main.py` after "GENERARE SHEET-URI CONSOLIDATE" (~line 567) |
| Legends | `main.py:legends` dict (~line 303) |
| Alert thresholds | `config.py:RECOMMENDATION_THRESHOLDS` |
| Consolidated sheet method | `report_generator.py:ExcelReportGenerator.add_consolidated_sheet()` |
| Consolidated page method | `report_generator.py:PDFReportGenerator.add_consolidated_page()` |
## Adding New Reports
1. Add SQL query constant in `queries.py`
2. Add entry to `QUERIES` dict with `sql`, `params`, `title`, `description`
3. Add query name to `sheet_order` list in `main.py` (line ~143)
4. For PDF inclusion, add rendering logic in `main.py:generate_reports()`
1. Add SQL constant in `queries.py` (e.g., `NEW_QUERY = """SELECT..."""`)
2. Add to `QUERIES` dict: `'new_query': {'sql': NEW_QUERY, 'params': {'months': 12}, 'title': '...', 'description': '...'}`
3. Add `'new_query'` to `sheet_order` in `main.py`
4. Add legend in `legends` dict if needed
5. For PDF: add rendering in PDF section of `generate_reports()`
## Alert Thresholds (in config.py)
## Adding Consolidated Views
- Low margin: < 15%
- Price variation: > 20%
- Slow stock: > 90 days without movement
- Minimum sales for analysis: 1000 RON
To add data to consolidated sheets, modify the `sections` list in `add_consolidated_sheet()` calls:
```python
excel_gen.add_consolidated_sheet(
name='Sheet Name',
sections=[
{'title': 'Section', 'df': results.get('query_name'), 'legend': legends.get('query_name')}
]
)
```
## Oracle Schema Conventions
- `sters = 0` excludes deleted records
- `tip NOT IN (7, 8, 9, 24)` excludes returns/credit notes
- Account `341`, `345` = own production; `301` = raw materials
- Required views: `fact_vfacturi2`, `fact_vfacturi_detalii`, `vnom_articole`, `vnom_parteneri`, `vstoc`, `vrul`
## YoY Query Pattern
When creating Year-over-Year comparison queries:
1. Use CTEs for current period (`ADD_MONTHS(TRUNC(SYSDATE), -12)` to `SYSDATE`)
2. Use CTEs for previous period (`ADD_MONTHS(TRUNC(SYSDATE), -24)` to `ADD_MONTHS(TRUNC(SYSDATE), -12)`)
3. Handle empty previous data with `NVL()` fallback to 0
4. Add `TREND` column with values like `'CRESTERE'`, `'SCADERE'`, `'STABIL'`, `'FARA DATE YOY'`
## Conditional Formatting Colors
| Status | Excel Fill | Meaning |
|--------|------------|---------|
| OK/Good | `#4ECDC4` (teal) | CRESTERE, IMBUNATATIRE, DIVERSIFICARE |
| Warning | `#FFE66D` (yellow) | ATENTIE |
| Alert | `#FF6B6B` (red) | ALERTA, SCADERE, DETERIORARE, CONCENTRARE |