vending_data_intelligence_report/PERFORMANCE_ANALYSIS.md

Performance Analysis - Data Intelligence Report Generator

Date: 2024-12-11 Total Runtime: ~33 minutes (1971 seconds)

Executive Summary

Category	Time	Percentage
SQL Queries	~31 min	94%
Excel Generation	~12 sec	0.6%
PDF Generation	~3 sec	0.2%
Other (consolidation, recommendations)	<1 sec	<0.1%

Conclusion: The bottleneck is 100% in Oracle SQL queries. Excel and PDF generation are negligible.

---

Top 20 Slowest Operations

Rank	Operation	Duration	Rows	Notes
1	QUERY: clienti_sub_medie	130.63s	100	Complex aggregation
2	QUERY: vanzari_lunare	129.90s	25	Monthly aggregation over 12 months
3	QUERY: indicatori_agregati_venituri_yoy	129.31s	3	YoY comparison - 24 month scan
4	QUERY: sumar_executiv_yoy	129.05s	5	YoY comparison - 24 month scan
5	QUERY: sumar_executiv	129.84s	6	Basic KPIs
6	QUERY: dispersie_preturi	97.11s	50	Price variance analysis
7	QUERY: trending_clienti	69.84s	12514	Large result set
8	QUERY: marja_per_client	68.58s	7760	Large result set
9	QUERY: concentrare_risc_yoy	66.33s	3	YoY comparison
10	QUERY: concentrare_risc	66.19s	3	Risk concentration
11	QUERY: dso_dpo_yoy	65.88s	2	YoY comparison
12	QUERY: clienti_marja_mica	65.93s	7	Low margin clients
13	QUERY: sezonalitate_lunara	65.93s	12	Seasonality
14	QUERY: concentrare_clienti	65.76s	31	Client concentration
15	QUERY: indicatori_agregati_venituri	65.59s	3	Revenue indicators
16	QUERY: marja_client_categorie	65.27s	2622	Client-category margins
17	QUERY: top_produse	65.26s	50	Top products
18	QUERY: clienti_ranking_profit	65.03s	2463	Client profit ranking
19	QUERY: marja_per_categorie	64.85s	4	Margin by category
20	QUERY: productie_vs_revanzare	64.86s	3	Production vs resale

---

Fast Queries (<5 seconds)

Query	Duration	Rows
stoc_lent	0.06s	100
solduri_furnizori	0.08s	172
pozitia_cash	0.10s	4
indicatori_lichiditate	0.13s	4
analiza_prajitorie	0.15s	39
stoc_curent	0.16s	28
solduri_clienti	0.29s	825
facturi_restante_furnizori	0.55s	100
dso_dpo	0.65s	2
ciclu_conversie_cash	0.95s	4
clasificare_datorii	0.99s	5
facturi_restante	1.24s	100
aging_datorii	1.43s	305
portofoliu_clienti	1.60s	5
rotatie_stocuri	1.70s	100
grad_acoperire_datorii	2.17s	5
proiectie_lichiditate	2.17s	4
aging_creante	4.37s	5281

---

Excel Generation Breakdown

Operation	Duration	Rows
Save workbook	4.12s	-
trending_clienti sheet	2.43s	12514
marja_per_client sheet	2.56s	7760
aging_creante sheet	1.57s	5281
clienti_ranking_profit sheet	0.78s	2463
marja_client_categorie sheet	0.56s	2622
All other sheets	<0.2s each	-

Total Excel: ~12 seconds

---

PDF Generation Breakdown

Operation	Duration
Chart: vanzari_lunare	0.80s
Chart: concentrare_clienti	0.61s
Chart: ciclu_conversie_cash	0.33s
Chart: productie_vs_revanzare	0.21s
Save document	0.49s
All pages	<0.01s each

Total PDF: ~3 seconds

---

Root Cause Analysis

Why are queries slow?

1. Full table scans on fact_vfacturi2

   • Most queries filter by data >= ADD_MONTHS(SYSDATE, -12) or -24
   • Without an index on data, Oracle scans the entire table

2. YoY queries scan 24 months

   • sumar_executiv_yoy, indicatori_agregati_venituri_yoy, etc.
   • These compare current 12 months vs previous 12 months
   • Double the data scanned

3. Complex JOINs without indexes

   • Joins between fact_vfacturi2, fact_vfacturi_detalii, vnom_articole, vnom_parteneri
   • Missing indexes on foreign keys

4. Repeated aggregations

   • Multiple queries calculate similar sums (vânzări, marjă)
   • Each query re-scans the same data

---

Optimization Recommendations

Priority 1: Add Indexes (Immediate Impact)

-- Index on date column (most critical)
CREATE INDEX idx_vfacturi2_data ON fact_vfacturi2(data);

-- Composite index for common filters
CREATE INDEX idx_vfacturi2_filter ON fact_vfacturi2(sters, tip, data);

-- Index on detail table join column
CREATE INDEX idx_vfacturi_det_nrfac ON fact_vfacturi_detalii(nrfactura);

Priority 2: Materialized Views (Medium-term)

-- Pre-aggregated monthly sales
CREATE MATERIALIZED VIEW mv_vanzari_lunare
BUILD IMMEDIATE
REFRESH COMPLETE ON DEMAND
AS
SELECT
    TRUNC(data, 'MM') as luna,
    SUM(valoare) as vanzari,
    SUM(marja) as marja
FROM fact_vfacturi2
WHERE sters = 0 AND tip NOT IN (7,8,9,24)
GROUP BY TRUNC(data, 'MM');

Priority 3: Query Consolidation (Long-term)

• Combine related queries into single CTEs
• Calculate base metrics once, derive others
• Use window functions instead of self-joins for YoY

---

Monitoring

Run with performance logging enabled:

python main.py --months 12
# Check output/performance_log.txt for detailed breakdown

---

Version History

Date	Change
2024-12-11	Initial performance analysis with PerformanceLogger