Implement Dashboard consolidation + Performance logging
Features: - Add unified "Dashboard Complet" sheet (Excel) with all 9 sections - Add unified "Dashboard Complet" page (PDF) with key metrics - Fix VALOARE_ANTERIOARA NULL bug (use sumar_executiv_yoy directly) - Add PerformanceLogger class for timing analysis - Remove redundant consolidated sheets (keep only Dashboard Complet) Bug fixes: - Fix Excel formula error (=== interpreted as formula, changed to >>>) - Fix args.output → args.output_dir in perf.summary() Performance analysis: - Add PERFORMANCE_ANALYSIS.md with detailed breakdown - SQL queries take 94% of runtime (31 min), Excel/PDF only 1% - Identified slow queries for optimization Documentation: - Update CLAUDE.md with new structure - Add context handover for query optimization task 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
179
PERFORMANCE_ANALYSIS.md
Normal file
179
PERFORMANCE_ANALYSIS.md
Normal file
@@ -0,0 +1,179 @@
|
||||
# Performance Analysis - Data Intelligence Report Generator
|
||||
|
||||
**Date:** 2024-12-11
|
||||
**Total Runtime:** ~33 minutes (1971 seconds)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Category | Time | Percentage |
|
||||
|----------|------|------------|
|
||||
| **SQL Queries** | ~31 min | **94%** |
|
||||
| Excel Generation | ~12 sec | 0.6% |
|
||||
| PDF Generation | ~3 sec | 0.2% |
|
||||
| Other (consolidation, recommendations) | <1 sec | <0.1% |
|
||||
|
||||
**Conclusion:** The bottleneck is 100% in Oracle SQL queries. Excel and PDF generation are negligible.
|
||||
|
||||
---
|
||||
|
||||
## Top 20 Slowest Operations
|
||||
|
||||
| Rank | Operation | Duration | Rows | Notes |
|
||||
|------|-----------|----------|------|-------|
|
||||
| 1 | `QUERY: clienti_sub_medie` | 130.63s | 100 | Complex aggregation |
|
||||
| 2 | `QUERY: vanzari_lunare` | 129.90s | 25 | Monthly aggregation over 12 months |
|
||||
| 3 | `QUERY: indicatori_agregati_venituri_yoy` | 129.31s | 3 | YoY comparison - 24 month scan |
|
||||
| 4 | `QUERY: sumar_executiv_yoy` | 129.05s | 5 | YoY comparison - 24 month scan |
|
||||
| 5 | `QUERY: sumar_executiv` | 129.84s | 6 | Basic KPIs |
|
||||
| 6 | `QUERY: dispersie_preturi` | 97.11s | 50 | Price variance analysis |
|
||||
| 7 | `QUERY: trending_clienti` | 69.84s | 12514 | Large result set |
|
||||
| 8 | `QUERY: marja_per_client` | 68.58s | 7760 | Large result set |
|
||||
| 9 | `QUERY: concentrare_risc_yoy` | 66.33s | 3 | YoY comparison |
|
||||
| 10 | `QUERY: concentrare_risc` | 66.19s | 3 | Risk concentration |
|
||||
| 11 | `QUERY: dso_dpo_yoy` | 65.88s | 2 | YoY comparison |
|
||||
| 12 | `QUERY: clienti_marja_mica` | 65.93s | 7 | Low margin clients |
|
||||
| 13 | `QUERY: sezonalitate_lunara` | 65.93s | 12 | Seasonality |
|
||||
| 14 | `QUERY: concentrare_clienti` | 65.76s | 31 | Client concentration |
|
||||
| 15 | `QUERY: indicatori_agregati_venituri` | 65.59s | 3 | Revenue indicators |
|
||||
| 16 | `QUERY: marja_client_categorie` | 65.27s | 2622 | Client-category margins |
|
||||
| 17 | `QUERY: top_produse` | 65.26s | 50 | Top products |
|
||||
| 18 | `QUERY: clienti_ranking_profit` | 65.03s | 2463 | Client profit ranking |
|
||||
| 19 | `QUERY: marja_per_categorie` | 64.85s | 4 | Margin by category |
|
||||
| 20 | `QUERY: productie_vs_revanzare` | 64.86s | 3 | Production vs resale |
|
||||
|
||||
---
|
||||
|
||||
## Fast Queries (<5 seconds)
|
||||
|
||||
| Query | Duration | Rows |
|
||||
|-------|----------|------|
|
||||
| `stoc_lent` | 0.06s | 100 |
|
||||
| `solduri_furnizori` | 0.08s | 172 |
|
||||
| `pozitia_cash` | 0.10s | 4 |
|
||||
| `indicatori_lichiditate` | 0.13s | 4 |
|
||||
| `analiza_prajitorie` | 0.15s | 39 |
|
||||
| `stoc_curent` | 0.16s | 28 |
|
||||
| `solduri_clienti` | 0.29s | 825 |
|
||||
| `facturi_restante_furnizori` | 0.55s | 100 |
|
||||
| `dso_dpo` | 0.65s | 2 |
|
||||
| `ciclu_conversie_cash` | 0.95s | 4 |
|
||||
| `clasificare_datorii` | 0.99s | 5 |
|
||||
| `facturi_restante` | 1.24s | 100 |
|
||||
| `aging_datorii` | 1.43s | 305 |
|
||||
| `portofoliu_clienti` | 1.60s | 5 |
|
||||
| `rotatie_stocuri` | 1.70s | 100 |
|
||||
| `grad_acoperire_datorii` | 2.17s | 5 |
|
||||
| `proiectie_lichiditate` | 2.17s | 4 |
|
||||
| `aging_creante` | 4.37s | 5281 |
|
||||
|
||||
---
|
||||
|
||||
## Excel Generation Breakdown
|
||||
|
||||
| Operation | Duration | Rows |
|
||||
|-----------|----------|------|
|
||||
| Save workbook | 4.12s | - |
|
||||
| trending_clienti sheet | 2.43s | 12514 |
|
||||
| marja_per_client sheet | 2.56s | 7760 |
|
||||
| aging_creante sheet | 1.57s | 5281 |
|
||||
| clienti_ranking_profit sheet | 0.78s | 2463 |
|
||||
| marja_client_categorie sheet | 0.56s | 2622 |
|
||||
| All other sheets | <0.2s each | - |
|
||||
|
||||
**Total Excel:** ~12 seconds
|
||||
|
||||
---
|
||||
|
||||
## PDF Generation Breakdown
|
||||
|
||||
| Operation | Duration |
|
||||
|-----------|----------|
|
||||
| Chart: vanzari_lunare | 0.80s |
|
||||
| Chart: concentrare_clienti | 0.61s |
|
||||
| Chart: ciclu_conversie_cash | 0.33s |
|
||||
| Chart: productie_vs_revanzare | 0.21s |
|
||||
| Save document | 0.49s |
|
||||
| All pages | <0.01s each |
|
||||
|
||||
**Total PDF:** ~3 seconds
|
||||
|
||||
---
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
### Why are queries slow?
|
||||
|
||||
1. **Full table scans on `fact_vfacturi2`**
|
||||
- Most queries filter by `data >= ADD_MONTHS(SYSDATE, -12)` or `-24`
|
||||
- Without an index on `data`, Oracle scans the entire table
|
||||
|
||||
2. **YoY queries scan 24 months**
|
||||
- `sumar_executiv_yoy`, `indicatori_agregati_venituri_yoy`, etc.
|
||||
- These compare current 12 months vs previous 12 months
|
||||
- Double the data scanned
|
||||
|
||||
3. **Complex JOINs without indexes**
|
||||
- Joins between `fact_vfacturi2`, `fact_vfacturi_detalii`, `vnom_articole`, `vnom_parteneri`
|
||||
- Missing indexes on foreign keys
|
||||
|
||||
4. **Repeated aggregations**
|
||||
- Multiple queries calculate similar sums (vânzări, marjă)
|
||||
- Each query re-scans the same data
|
||||
|
||||
---
|
||||
|
||||
## Optimization Recommendations
|
||||
|
||||
### Priority 1: Add Indexes (Immediate Impact)
|
||||
|
||||
```sql
|
||||
-- Index on date column (most critical)
|
||||
CREATE INDEX idx_vfacturi2_data ON fact_vfacturi2(data);
|
||||
|
||||
-- Composite index for common filters
|
||||
CREATE INDEX idx_vfacturi2_filter ON fact_vfacturi2(sters, tip, data);
|
||||
|
||||
-- Index on detail table join column
|
||||
CREATE INDEX idx_vfacturi_det_nrfac ON fact_vfacturi_detalii(nrfactura);
|
||||
```
|
||||
|
||||
### Priority 2: Materialized Views (Medium-term)
|
||||
|
||||
```sql
|
||||
-- Pre-aggregated monthly sales
|
||||
CREATE MATERIALIZED VIEW mv_vanzari_lunare
|
||||
BUILD IMMEDIATE
|
||||
REFRESH COMPLETE ON DEMAND
|
||||
AS
|
||||
SELECT
|
||||
TRUNC(data, 'MM') as luna,
|
||||
SUM(valoare) as vanzari,
|
||||
SUM(marja) as marja
|
||||
FROM fact_vfacturi2
|
||||
WHERE sters = 0 AND tip NOT IN (7,8,9,24)
|
||||
GROUP BY TRUNC(data, 'MM');
|
||||
```
|
||||
|
||||
### Priority 3: Query Consolidation (Long-term)
|
||||
|
||||
- Combine related queries into single CTEs
|
||||
- Calculate base metrics once, derive others
|
||||
- Use window functions instead of self-joins for YoY
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
Run with performance logging enabled:
|
||||
```bash
|
||||
python main.py --months 12
|
||||
# Check output/performance_log.txt for detailed breakdown
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2024-12-11 | Initial performance analysis with PerformanceLogger |
|
||||
Reference in New Issue
Block a user