roa2web-service-auto/DIAGNOSIS-2025-12-30.md

ROA2WEB Production Deployment Issue - Diagnostic Report

Date: 2025-12-30 Environment: Windows Production (roa2web.romfast.ro) Issue: 404 errors on API calls after deployment

---

Summary

Frontend loads successfully but all API calls return 404 Not Found errors:

• POST /roa2web/api/auth/login → 404
• GET /roa2web/api/companies → 404

Browser console shows:

POST https://roa2web.romfast.ro/roa2web/api/auth/login 404 (Not Found)

---

Architecture Overview

2-Tier IIS Deployment:

Internet → Public IIS (10.0.20.122 - roa2web.romfast.ro)
              ↓ HTTPS reverse proxy
          Internal IIS (10.0.20.36 - application server)
              ↓ API proxy to localhost
          Backend Service (localhost:8000 on 10.0.20.36)
              ↓
          Oracle Database

Verified Components

✅ Public IIS (10.0.20.122 - roa2web.romfast.ro):

• ✅ web.config correctly proxies all requests to https://10.0.20.36/{R:1}
• ✅ Forwards headers: X-Forwarded-Proto, X-Forwarded-Host, X-Real-IP
• ✅ Configuration verified - THIS SERVER IS CORRECT

✅ Frontend (10.0.20.36):

• Built with base: '/roa2web/' (correct for IIS sub-application)
• API calls use baseURL: '/roa2web/api' (correct)
• Static files load successfully

❌ Internal IIS (10.0.20.36) - PROBLEMA ESTE AICI:

• CRITICAL: web.config lipsește SAU nu este configurat corect
• MUST: Trebuie să facă proxy de la /roa2web/api/* către http://localhost:8000/api/*
• Location: C:\inetpub\wwwroot\roa2web\web.config (pe serverul 10.0.20.36)

❓ Backend Service (pe 10.0.20.36):

• UNKNOWN: Need to verify service is running
• Should be accessible at http://localhost:8000/health (de pe 10.0.20.36)

---

Root Cause Analysis

Based on Playwright testing, the issue is most likely:

Primary Suspect: Internal Server web.config

The internal server's web.config MUST handle the /roa2web/ prefix correctly.

WRONG Configuration (causes 404):

<rule name="Proxy Unified API" stopProcessing="true">
  <match url="^api/(.*)" />  ❌ Missing roa2web prefix!
  <action type="Rewrite" url="http://localhost:8000/api/{R:1}" />
</rule>

CORRECT Configuration:

<rule name="Proxy Unified API" stopProcessing="true">
  <match url="^roa2web/api/(.*)" />  ✅ Includes roa2web prefix
  <action type="Rewrite" url="http://localhost:8000/api/{R:1}" />
</rule>

Why: Requests arrive at internal server as /roa2web/api/auth/login (NOT /api/auth/login), because the public server preserves the full path when proxying.

Secondary Suspects

1. Backend service not running

   • Check: Get-Service ROA2WEB-Backend

2. IIS ARR not enabled

   • Check: Get-WebConfigurationProperty -PSPath "MACHINE/WEBROOT/APPHOST" -Filter "system.webServer/proxy" -Name "enabled"

3. IIS sub-application misconfigured

   • Frontend should be deployed at IIS path /roa2web, not root

---

Diagnostic Steps

Step 1: Check Backend Service (on 10.0.20.36)

# Check service status
Get-Service ROA2WEB-Backend

# If stopped, start it
Start-Service ROA2WEB-Backend

# Test backend directly
Invoke-WebRequest http://localhost:8000/health

Expected Output:

{"status": "healthy", "version": "..."}

Step 2: Check Internal IIS web.config (on 10.0.20.36)

# View current web.config
Get-Content C:\inetpub\wwwroot\roa2web\web.config

# OR if frontend is in subdirectory:
Get-Content C:\inetpub\wwwroot\roa2web\frontend\web.config

Verify these rules:

• <match url="^roa2web/api/(.*)" /> (NOT ^api/(.*))
• <match url="^roa2web/uploads/(.*)" /> (NOT ^uploads/(.*))
• <match url="^roa2web/.*" /> for SPA fallback

Step 3: Check IIS ARR (on 10.0.20.36)

# Check if ARR is installed
Get-WindowsFeature -Name Web-ARR

# Check if proxy is enabled
Get-WebConfigurationProperty -PSPath "MACHINE/WEBROOT/APPHOST" `
    -Filter "system.webServer/proxy" `
    -Name "enabled"

Expected: enabled should be True

Step 4: Test Internal IIS Proxy (on 10.0.20.36)

# Test API endpoint through IIS
Invoke-WebRequest https://localhost/roa2web/api/health

# Should proxy to backend and return response

Step 5: Check IIS Application Structure (on 10.0.20.36)

# List IIS applications
Get-WebApplication -Site "Default Web Site" | Format-Table Path, PhysicalPath

# Should show:
# /roa2web -> C:\inetpub\wwwroot\roa2web\

---

Fix Recommendations

Fix #1: Update Internal Server web.config

Server: 10.0.20.36 (internal application server) Location: C:\inetpub\wwwroot\roa2web\web.config

IMPORTANT: Acest web.config este pe serverul 10.0.20.36, NU pe 10.0.20.122!

Required Configuration:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <!-- Proxy API requests to unified backend -->
        <rule name="Proxy Unified API" stopProcessing="true">
          <match url="^roa2web/api/(.*)" />
          <action type="Rewrite" url="http://localhost:8000/api/{R:1}" />
        </rule>

        <!-- Proxy uploads to unified backend -->
        <rule name="Proxy Uploads" stopProcessing="true">
          <match url="^roa2web/uploads/(.*)" />
          <action type="Rewrite" url="http://localhost:8000/uploads/{R:1}" />
        </rule>

        <!-- SPA fallback - serve index.html for non-file requests -->
        <rule name="SPA Fallback" stopProcessing="true">
          <match url="^roa2web/.*" />
          <conditions logicalGrouping="MatchAll">
            <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
            <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
          </conditions>
          <action type="Rewrite" url="/roa2web/index.html" />
        </rule>
      </rules>
    </rewrite>

    <staticContent>
      <mimeMap fileExtension=".webmanifest" mimeType="application/manifest+json" />
      <mimeMap fileExtension=".js" mimeType="application/javascript" />
      <mimeMap fileExtension=".json" mimeType="application/json" />
    </staticContent>

    <httpProtocol>
      <customHeaders>
        <add name="Cache-Control" value="public, max-age=31536000" />
      </customHeaders>
    </httpProtocol>
  </system.webServer>
</configuration>

Deployment:

# ⚠️ IMPORTANT: Rulează aceste comenzi pe serverul 10.0.20.36, NU pe 10.0.20.122!
# Conectează-te la 10.0.20.36 (internal application server)

# Backup current config
Copy-Item C:\inetpub\wwwroot\roa2web\web.config `
          C:\inetpub\wwwroot\roa2web\web.config.backup.$(Get-Date -Format 'yyyyMMdd-HHmmss')

# Update web.config with corrected version
# (copy from repository: public/web.config after updating it)

# Restart IIS
iisreset

# Test
Invoke-WebRequest https://localhost/roa2web/api/health

Fix #2: Ensure Backend Service is Running

# On internal server (10.0.20.36)

# Check service
Get-Service ROA2WEB-Backend

# If stopped
Start-Service ROA2WEB-Backend

# View startup logs
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stdout.log -Tail 50

# Check for errors
Get-Content C:\inetpub\wwwroot\roa2web\logs\backend-stderr.log -Tail 50

Fix #3: Enable IIS ARR (if not enabled)

# On internal server (10.0.20.36)

# Enable proxy
Set-WebConfigurationProperty -PSPath "MACHINE/WEBROOT/APPHOST" `
    -Filter "system.webServer/proxy" `
    -Name "enabled" `
    -Value "True"

# Restart IIS
iisreset

---

Verification Steps (After Fix)

1. Backend Health Check (Direct)

# On 10.0.20.36
Invoke-WebRequest http://localhost:8000/health

Expected: 200 OK with JSON response

2. Internal IIS Proxy Test

# On 10.0.20.36
Invoke-WebRequest https://localhost/roa2web/api/health

Expected: 200 OK (proxied from backend)

3. Public Access Test

# From any client
Invoke-WebRequest https://roa2web.romfast.ro/roa2web/

Expected: Frontend loads (HTML page)

4. Login Flow Test (Playwright)

Use Playwright to test complete login flow:

# On development machine
cd /mnt/e/proiecte/roa2web
npx playwright test --headed --project=chromium tests/login.spec.js

Expected: Login succeeds without 404 errors

---

Prevention

To prevent this issue in future deployments:

1. Use Correct web.config Template

   • Repository should have separate web.config templates:
     • public/web.config - for internal server (with /roa2web/ prefix)
     • deployment/windows/config/web.config - same as above

2. Automated Deployment Script

   • PowerShell script should validate web.config before deployment
   • Check for required URL patterns: ^roa2web/api/, ^roa2web/uploads/

3. Deployment Checklist

   • Add to deployment/windows/docs/DEPLOYMENT-CHECKLIST.md:
     • Verify web.config includes /roa2web/ prefix in match rules
     • Test backend: Invoke-WebRequest http://localhost:8000/health
     • Test IIS proxy: Invoke-WebRequest https://localhost/roa2web/api/health
     • Test public access: Invoke-WebRequest https://roa2web.romfast.ro/roa2web/

4. Monitoring

   • Add health check endpoint monitoring
   • Alert if backend service stops
   • Monitor IIS logs for 404/502 errors

---

Next Steps

1. ✅ Immediate: Check internal server web.config (on 10.0.20.36)
2. ✅ Immediate: Verify backend service is running
3. ✅ Immediate: Apply Fix #1 if web.config is incorrect
4. ⏭️ After Fix: Run verification steps
5. ⏭️ After Fix: Update repository web.config template
6. ⏭️ Future: Add automated deployment validation

---

Documentation Created

• ✅ deployment/windows/docs/TWO-TIER-IIS-DEPLOYMENT.md - Complete 2-tier architecture guide
• ✅ Updated CLAUDE.md with deployment documentation reference
• ✅ This diagnostic report: DIAGNOSIS-2025-12-30.md

---

Contact Information

Issue Reported By: User via Playwright testing Diagnosed By: Claude Code Date: 2025-12-30 Severity: HIGH (blocking production login) Priority: IMMEDIATE

---

For detailed architecture documentation, see: deployment/windows/docs/TWO-TIER-IIS-DEPLOYMENT.md