ROMFASTSQL/oracle/standby-server-scripts/DR_UPGRADE_TO_CUMULATIVE_PLAN.md

Oracle DR - Upgrade to Cumulative Incremental Backup Strategy

Generated: 2025-10-09 Last Updated: 2025-10-10 00:45 Status: 🟡 IN PROGRESS - Phases 1-3-5 COMPLETED Objective: Implement cumulative incremental backups with Proxmox host storage for optimal RPO/RTO Target RPO: 3-4 hours (vs current 24 hours) Target RTO: 12-15 minutes (unchanged)

---

✅ IMPLEMENTATION STATUS

Completed (2025-10-09 + 2025-10-10)

• ✅ Phase 1: Proxmox host storage configured (/mnt/pve/oracle-backups/ROA/autobackup)
• ✅ Phase 2: RMAN script already has CUMULATIVE keyword
• ✅ Phase 3: Transfer scripts updated to send to Proxmox (10.0.20.202:22, root)
  • Modified: transfer_incremental.ps1 and transfer_to_dr.ps1
  • Changed from VM 109 (10.0.20.37:22122) to Proxmox host
  • Converted Windows PowerShell commands to Linux bash
• ✅ VM 109 cleanup: Deleted temporary files, old backups (~6.4 GB freed)
• ✅ Phase 5: NFS mount point configured on VM 109 → F:\ drive
  • NFS server installed on Proxmox: nfs-kernel-server
  • NFS export configured: /mnt/pve/oracle-backups → 10.0.20.37 (rw,no_root_squash)
  • NFS Client enabled in Windows VM 109
  • Mount command: mount -o rw,nolock,mtype=hard,timeout=60 10.0.20.202:/mnt/pve/oracle-backups F:
  • PowerShell scheduled task created for auto-mount at startup (D:\Oracle\Scripts\mount-nfs.bat)
  • Permissions set to 777 on Proxmox directory
  • Status: F:\ mounts automatically at Windows startup ✅

Pending (Next Session)

• ⏳ SSH Key Setup: Run copy_existing_key_to_proxmox.ps1 on PRIMARY as Administrator
  • Existing key: C:\Windows\System32\config\systemprofile\.ssh\id_rsa
  • Copy to: Proxmox /root/.ssh/authorized_keys
• ⏳ Phase 4: Modify scheduled tasks on PRIMARY (13:00 + 18:00)
• ⏳ Phase 6: Update restore script to use F:\ mount
• ⏳ Phase 7: Test FULL + CUMULATIVE backup and restore

Files Modified

oracle/standby-server-scripts/
├── transfer_incremental.ps1          [MODIFIED] → Proxmox host
├── transfer_to_dr.ps1                [MODIFIED] → Proxmox host
├── rman_backup_incremental.txt       [ALREADY OK] → Has CUMULATIVE
└── copy_existing_key_to_proxmox.ps1  [NEW] → Setup script for SSH key

VM 109 (Windows):
├── C:\Scripts\mount-nfs.ps1          [NEW] → PowerShell script for NFS mount
└── Scheduled Task: "Mount NFS F"     [NEW] → Auto-mount at startup

Proxmox (pveelite):
├── /etc/exports                      [MODIFIED] → NFS export configuration
└── /mnt/pve/oracle-backups/          [PERMISSIONS] → chmod 777

---

📋 EXECUTIVE SUMMARY

Current State

• Backup Strategy: FULL daily (02:30), DIFFERENTIAL incremental (14:00)
• Storage: Backups transferred to VM 109 (powered OFF most of time)
• RPO: 24 hours (only FULL backup used for restore)
• Issue: DIFFERENTIAL incremental caused UNDO corruption during restore

Proposed State

• Backup Strategy: FULL daily (02:30), CUMULATIVE incremental (13:00 + 18:00)
• Storage: Backups on Proxmox host (pveelite), mounted in VM 109 when needed
• RPO: 3-4 hours (using FULL + latest CUMULATIVE)
• Benefit: Simple, reliable restore without UNDO/SCN issues

Why CUMULATIVE?

• ✅ Simple restore: FULL + last cumulative (no dependency chain)
• ✅ No UNDO corruption: Each cumulative is independent from Level 0
• ✅ Better RPO: Max 5 hours data loss (vs 24 hours)
• ✅ Reliable: No issues with missing intermediate backups

---

🎯 IMPLEMENTATION PHASES

PHASE 1: Configure Proxmox Host Storage (15 minutes)

Objective: Create backup storage on pveelite host, accessible by VM 109 via mount point

Steps:

1.1 Create backup directory on pveelite (SSH to host)

# On pveelite (10.0.20.202)
ssh root@10.0.20.202

# Create directory structure
mkdir -p /mnt/pve/oracle-backups/ROA/autobackup
chmod 755 /mnt/pve/oracle-backups
chmod 755 /mnt/pve/oracle-backups/ROA
chmod 755 /mnt/pve/oracle-backups/ROA/autobackup

# Verify
ls -la /mnt/pve/oracle-backups/ROA/autobackup

1.2 Add mount point to VM 109 (Proxmox CLI)

# Stop VM 109 if running
qm stop 109

# Add mount point as additional storage
# This creates a VirtIO-9p mount point
qm set 109 -mp0 /mnt/pve/oracle-backups,mp=/mnt/oracle-backups

# Or via Proxmox Web UI:
# VM 109 → Hardware → Add → Mount Point
# - Source: /mnt/pve/oracle-backups
# - Mount point: /mnt/oracle-backups
# - Read-only: NO

# Start VM to test
qm start 109

1.3 Verify mount in Windows VM

# SSH to VM 109
ssh -p 22122 romfast@10.0.20.37

# Check if mount point appears as drive
# ⚠️ IMPORTANT: E:\ is already used in VM 109
# Mount will appear as F:\ (next available drive letter)
Get-PSDrive -PSProvider FileSystem

# Expected: C:, D:, E: (existing), F: (new mount from host)

# Verify mount path accessible
Test-Path F:\ROA\autobackup

# Create test file
New-Item -ItemType Directory -Path F:\ROA\autobackup -Force
echo "test" > F:\ROA\autobackup\test.txt

# Verify from host
exit
ssh root@10.0.20.202 "ls -la /mnt/pve/oracle-backups/ROA/autobackup/test.txt"

# Should show the test file - mount is working!

⚠️ CRITICAL NOTE:

• VM 109 already has E:\ partition
• Mount point will be *F:* (not E:)
• Update all scripts to use *F:* instead of E:\

---

PHASE 2: Modify RMAN Backup Scripts on PRIMARY (20 minutes)

Objective: Change incremental backups from DIFFERENTIAL to CUMULATIVE, add second daily incremental

2.1 Găsește scriptul RMAN incremental existent

# SSH to PRIMARY
ssh -p 22122 Administrator@10.0.20.36

cd D:\rman_backup

# Găsește scriptul incremental existent
Get-ChildItem *incr*.txt, *incr*.rman

# Ar trebui să vezi ceva gen:
# rman_backup_incremental.txt SAU
# rman_incremental.rman SAU similar

2.2 Modifică scriptul EXISTENT - adaugă doar un cuvânt

Fișier: Scriptul incremental găsit la pasul 2.1 (ex: D:\rman_backup\rman_backup_incremental.txt)

Modificare: Găsește linia cu INCREMENTAL LEVEL 1 și adaugă CUMULATIVE

ÎNAINTE:

BACKUP INCREMENTAL LEVEL 1 ...

DUPĂ:

BACKUP INCREMENTAL LEVEL 1 CUMULATIVE ...

Asta e tot! Un singur cuvânt adăugat.

Exemplu complet (dacă scriptul arată așa):

ÎNAINTE:
BACKUP INCREMENTAL LEVEL 1 AS COMPRESSED BACKUPSET DATABASE ...

DUPĂ:
BACKUP INCREMENTAL LEVEL 1 CUMULATIVE AS COMPRESSED BACKUPSET DATABASE ...

2.3 Test manual

# On PRIMARY
cd D:\rman_backup

# Rulează scriptul modificat
# Folosește numele scriptului tău existent!
rman cmdfile=rman_backup_incremental.txt log=logs\test_cumulative_$(Get-Date -Format 'yyyyMMdd_HHmmss').log

# Verifică că s-a creat backup
Get-ChildItem C:\Users\oracle\recovery_area\ROA\autobackup\*.bkp | Sort-Object LastWriteTime -Descending | Select-Object -First 3

---

PHASE 3: Update Transfer Scripts (30 minutes)

Objective: Update transfer scripts to send backups to Proxmox host instead of VM

3.1 Găsește scripturile de transfer existente

# SSH to PRIMARY
ssh -p 22122 Administrator@10.0.20.36

cd D:\rman_backup

# Găsește scripturile de transfer
Get-ChildItem *transfer*.ps1

# Ar trebui să vezi:
# - transfer_to_dr.ps1 (pentru FULL)
# - transfer_incremental.ps1 SAU 02b_transfer_incremental_to_dr.ps1 (pentru INCREMENTAL)

3.2 Modifică scripturile EXISTENTE - schimbă doar destinația

Găsește în fiecare script aceste linii și modifică-le:

ÎNAINTE (transfer la VM):

$DRHost = "10.0.20.37"       # VM-ul
$DRPort = "22122"            # SSH pe VM
$DRUser = "romfast"          # User din VM
$DRPath = "D:/oracle/backups/primary"  # Path în VM

DUPĂ (transfer la Proxmox host):

$DRHost = "10.0.20.202"      # pveelite HOST
$DRPort = "22"               # SSH standard pe host
$DRUser = "root"             # Root pe Proxmox
$DRPath = "/mnt/pve/oracle-backups/ROA/autobackup"  # Path pe host

Asta e tot! Doar 4 linii modificate în fiecare script.

3.2 Setup SSH key for Proxmox host access

# On PRIMARY (10.0.20.36)

# Generate SSH key for Proxmox host (if not exists)
ssh-keygen -t rsa -b 4096 -f C:\Users\Administrator\.ssh\id_rsa_pveelite -N ""

# Copy public key to Proxmox host
type C:\Users\Administrator\.ssh\id_rsa_pveelite.pub | ssh root@10.0.20.202 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

# Test connection
ssh -i C:\Users\Administrator\.ssh\id_rsa_pveelite root@10.0.20.202 "echo SSH_OK"

3.3 Test transfer script

# On PRIMARY
cd D:\rman_backup

# Test FULL backup transfer
.\02_transfer_to_pveelite_host.ps1 -BackupType FULL

# Verify on Proxmox host
ssh root@10.0.20.202 "ls -lh /mnt/pve/oracle-backups/ROA/autobackup/*.bkp"

# Test INCREMENTAL backup transfer
.\02_transfer_to_pveelite_host.ps1 -BackupType INCREMENTAL

---

PHASE 4: Update Scheduled Tasks on PRIMARY (20 minutes)

Objective: Create/update scheduled tasks for 2 cumulative incremental backups per day

4.1 View current scheduled tasks

# On PRIMARY
Get-ScheduledTask | Where-Object {$_.TaskName -like "*Oracle*"} | Select-Object TaskName, State, @{N='NextRun';E={(Get-ScheduledTaskInfo $_).NextRunTime}}

4.2 Găsește task-ul incremental existent (14:00)

# On PRIMARY
Get-ScheduledTask | Where-Object {$_.TaskName -like "*incr*" -or $_.TaskName -like "*14*"} | Select-Object TaskName, State

# Notează numele exact al task-ului

4.3 Modifică task-ul 14:00 → 13:00 (primul incremental)

# Folosește numele găsit mai sus
$taskName = "Oracle RMAN Incremental Backup"  # ÎNLOCUIEȘTE cu numele real!

# Schimbă doar ora: 14:00 → 13:00
$trigger = New-ScheduledTaskTrigger -Daily -At "13:00"

$task = Get-ScheduledTask -TaskName $taskName
Set-ScheduledTask -TaskName $taskName -Trigger $trigger

4.4 Clonează task-ul pentru al doilea incremental (18:00)

# Exportă task-ul existent
$task = Get-ScheduledTask -TaskName $taskName
$xml = [xml](Export-ScheduledTask -TaskName $taskName)

# Modifică ora în XML
$xml.Task.Triggers.CalendarTrigger.StartBoundary = $xml.Task.Triggers.CalendarTrigger.StartBoundary -replace "T13:00:", "T18:00:"

# Importă ca task nou
Register-ScheduledTask -TaskName "$taskName 1800" -Xml $xml.OuterXml

# Sau mai simplu - copiază task-ul din Task Scheduler GUI și schimbă ora

4.5 Verifică toate task-urile

# Ar trebui să vezi 3 task-uri Oracle:
# 1. FULL (02:30) - neschimbat
# 2. INCREMENTAL (13:00) - modificat din 14:00
# 3. INCREMENTAL (18:00) - clonat din 13:00

Get-ScheduledTask | Where-Object {$_.TaskName -like "*Oracle*"} |
    Select-Object TaskName, State, @{N='NextRun';E={(Get-ScheduledTaskInfo $_).NextRunTime}} |
    Format-Table -AutoSize

4.5 Verify all tasks

# List all Oracle tasks
Get-ScheduledTask | Where-Object {$_.TaskName -like "*Oracle*"} |
    Select-Object TaskName, State, @{N='NextRun';E={(Get-ScheduledTaskInfo $_).NextRunTime}} |
    Format-Table -AutoSize

# Expected tasks:
# 1. Oracle RMAN Full Backup 0230       - Daily 02:30
# 2. Oracle RMAN Cumulative Backup 1300 - Daily 13:00
# 3. Oracle RMAN Cumulative Backup 1800 - Daily 18:00

---

PHASE 5: Configure NFS Mount Point on VM 109 (30 minutes) ✅ COMPLETED

Objective: Mount Proxmox backup storage as F:\ drive in Windows VM using NFS

Status: ✅ COMPLETED on 2025-10-10

5.1 Install and configure NFS server on Proxmox

# SSH to Proxmox host
ssh root@10.0.20.202

# Install NFS server
apt install -y nfs-kernel-server

# Configure NFS export
echo '/mnt/pve/oracle-backups 10.0.20.37(rw,sync,no_subtree_check,no_root_squash)' >> /etc/exports

# Apply export configuration
exportfs -ra

# Set permissions for Windows compatibility
chmod -R 777 /mnt/pve/oracle-backups

# Verify export
showmount -e localhost
# Expected output: /mnt/pve/oracle-backups 10.0.20.37

5.2 Enable NFS Client in Windows VM 109

# SSH to VM 109
ssh -p 22122 romfast@10.0.20.37

# Enable NFS Client feature
Enable-WindowsOptionalFeature -Online -FeatureName ServicesForNFS-ClientOnly -All -NoRestart
Enable-WindowsOptionalFeature -Online -FeatureName ClientForNFS-Infrastructure -All -NoRestart
Enable-WindowsOptionalFeature -Online -FeatureName NFS-Administration -All -NoRestart

# Verify installation
Get-WindowsOptionalFeature -Online | Where-Object {$_.FeatureName -like "*NFS*"}

5.3 Create PowerShell mount script with auto-retry

# Create Scripts directory
mkdir C:\Scripts

# Create mount script
notepad C:\Scripts\mount-nfs.ps1

Content of C:\Scripts\mount-nfs.ps1:

Start-Sleep -Seconds 10

# Wait for NFS Client service
$timeout = 60
$elapsed = 0
while ($elapsed -lt $timeout) {
    $nfsService = Get-Service | Where-Object {$_.Name -like "*NFS*" -and $_.Status -eq "Running"}
    if ($nfsService) { break }
    Start-Sleep -Seconds 5
    $elapsed += 5
}

# Unmount F: if exists
try { & umount F: 2>$null } catch {}

# Mount NFS share
Start-Sleep -Seconds 5
& mount -o rw,nolock,mtype=hard,timeout=60 10.0.20.202:/mnt/pve/oracle-backups F:

# Log result
"$(Get-Date) - Mount completed" | Out-File C:\Scripts\mount-nfs.log -Append

5.4 Create scheduled task for auto-mount at startup

# Create scheduled task (run in CMD as Administrator)
schtasks /create /tn "Mount NFS F" /tr "powershell.exe -ExecutionPolicy Bypass -File C:\Scripts\mount-nfs.ps1" /sc onstart /ru SYSTEM /rl HIGHEST /delay 0000:30 /f

# Verify task creation
schtasks /query /tn "Mount NFS F"

# Test manual mount
mount -o rw,nolock,mtype=hard,timeout=60 10.0.20.202:/mnt/pve/oracle-backups F:

# Verify mount
dir F:\ROA\autobackup

5.5 Verification checklist

• NFS server running on Proxmox (port 2049)
• Export visible: showmount -e 10.0.20.202
• Windows NFS Client services enabled
• F:\ drive mounts successfully with manual command
• Scheduled task runs at startup
• F:\ persists after VM reboot
• Can create/read/write files on F:\ROA\autobackup

⚠️ IMPORTANT NOTES:

• NFS uses IP-based authentication (no username/password)
• Only VM 109 (10.0.20.37) can access the share
• no_root_squash allows Windows to write as root
• Permissions 777 on Proxmox ensure Windows compatibility
• Mount point is *F:* (not E:, which is already in use)

---

PHASE 6: Update DR Restore Script (30 minutes)

Objective: Update restore script to read backups from F:\ mount point and handle cumulative backups

6.1 Modifică scriptul de restore existent pentru cumulative backups

Fișier: D:\oracle\scripts\rman_restore_final.cmd (scriptul tău existent)

Modificări necesare:

1. Schimbă locația backup-urilor:

REM ÎNAINTE:
set BACKUP_DIR=C:/Users/oracle/recovery_area/ROA/autobackup

REM DUPĂ (⚠️ F:\ nu E:\ - E:\ e deja folosit în VM!):
set BACKUP_DIR=F:/ROA/autobackup

2. Verifică că mount point-ul e accesibil: Adaugă la început:

REM Verifică mount point
if not exist F:\ROA\autobackup (
    echo ERROR: Mount point F:\ not accessible!
    echo Make sure VM has mount point configured and host is reachable
    exit /b 1
)
set PFILE=C:\Users\oracle\admin\ROA\pfile\initROA.ora
set LOG_FILE=D:\oracle\logs\restore_cumulative_%date:~-4%%date:~3,2%%date:~0,2%_%time:~0,2%%time:~3,2%%time:~6,2%.log

echo ============================================================================
echo Oracle DR Restore - FULL + CUMULATIVE Incremental
echo ============================================================================
echo DBID: %DBID%
echo Backup Location: %BACKUP_DIR% (mount from Proxmox host)
echo Log: %LOG_FILE%
echo ============================================================================

REM Step 1: Shutdown database if running
echo.
echo [STEP 1/8] Shutting down database...
echo SHUTDOWN ABORT; > D:\oracle\temp\shutdown.sql
echo EXIT; >> D:\oracle\temp\shutdown.sql
sqlplus / as sysdba @D:\oracle\temp\shutdown.sql 2>nul
timeout /t 5 /nobreak >nul

REM Step 2: Startup NOMOUNT
echo.
echo [STEP 2/8] Starting instance NOMOUNT...
echo STARTUP NOMOUNT PFILE='%PFILE%'; > D:\oracle\temp\nomount.sql
echo EXIT; >> D:\oracle\temp\nomount.sql
sqlplus / as sysdba @D:\oracle\temp\nomount.sql

if %errorlevel% neq 0 (
    echo ERROR: Failed to startup NOMOUNT
    exit /b 1
)

REM Step 3: Restore control file
echo.
echo [STEP 3/8] Restoring control file...
echo SET DBID %DBID%; > D:\oracle\temp\restore_ctl.rman
echo. >> D:\oracle\temp\restore_ctl.rman
echo RUN { >> D:\oracle\temp\restore_ctl.rman
echo     ALLOCATE CHANNEL ch1 DEVICE TYPE DISK; >> D:\oracle\temp\restore_ctl.rman
echo     # Find latest control file backup >> D:\oracle\temp\restore_ctl.rman
echo     RESTORE CONTROLFILE FROM '%BACKUP_DIR%/ctl*.bkp'; >> D:\oracle\temp\restore_ctl.rman
echo     RELEASE CHANNEL ch1; >> D:\oracle\temp\restore_ctl.rman
echo } >> D:\oracle\temp\restore_ctl.rman
echo EXIT; >> D:\oracle\temp\restore_ctl.rman

rman target / cmdfile=D:\oracle\temp\restore_ctl.rman

if %errorlevel% neq 0 (
    echo ERROR: Control file restore failed
    exit /b 1
)

REM Step 4: Mount database
echo.
echo [STEP 4/8] Mounting database...
echo ALTER DATABASE MOUNT; > D:\oracle\temp\mount.sql
echo EXIT; >> D:\oracle\temp\mount.sql
sqlplus / as sysdba @D:\oracle\temp\mount.sql

REM Step 5: Catalog all backups
echo.
echo [STEP 5/8] Cataloging backups from mount point...
echo CATALOG START WITH '%BACKUP_DIR%/' NOPROMPT; > D:\oracle\temp\catalog.rman
echo LIST BACKUP SUMMARY; >> D:\oracle\temp\catalog.rman
echo EXIT; >> D:\oracle\temp\catalog.rman

rman target / cmdfile=D:\oracle\temp\catalog.rman

REM Step 6: Restore and recover database
echo.
echo [STEP 6/8] Restoring FULL + latest CUMULATIVE...
echo RUN { > D:\oracle\temp\restore_db.rman
echo     ALLOCATE CHANNEL ch1 DEVICE TYPE DISK; >> D:\oracle\temp\restore_db.rman
echo     ALLOCATE CHANNEL ch2 DEVICE TYPE DISK; >> D:\oracle\temp\restore_db.rman
echo. >> D:\oracle\temp\restore_db.rman
echo     # RMAN will automatically select: >> D:\oracle\temp\restore_db.rman
echo     #   1. Level 0 (FULL from 02:30) >> D:\oracle\temp\restore_db.rman
echo     #   2. Latest Level 1 CUMULATIVE (from 13:00 or 18:00) >> D:\oracle\temp\restore_db.rman
echo. >> D:\oracle\temp\restore_db.rman
echo     RESTORE DATABASE; >> D:\oracle\temp\restore_db.rman
echo     RECOVER DATABASE; >> D:\oracle\temp\restore_db.rman
echo. >> D:\oracle\temp\restore_db.rman
echo     RELEASE CHANNEL ch1; >> D:\oracle\temp\restore_db.rman
echo     RELEASE CHANNEL ch2; >> D:\oracle\temp\restore_db.rman
echo } >> D:\oracle\temp\restore_db.rman
echo EXIT; >> D:\oracle\temp\restore_db.rman

rman target / cmdfile=D:\oracle\temp\restore_db.rman

if %errorlevel% neq 0 (
    echo ERROR: Database restore/recovery failed
    exit /b 1
)

REM Step 7: Open database with RESETLOGS
echo.
echo [STEP 7/8] Opening database with RESETLOGS...
echo ALTER DATABASE OPEN RESETLOGS; > D:\oracle\temp\open.sql
echo EXIT; >> D:\oracle\temp\open.sql
sqlplus / as sysdba @D:\oracle\temp\open.sql

REM Step 8: Create TEMP and verify
echo.
echo [STEP 8/8] Creating TEMP tablespace and verifying...
echo ALTER TABLESPACE TEMP ADD TEMPFILE 'C:\Users\oracle\oradata\ROA\temp01.dbf' > D:\oracle\temp\verify.sql
echo   SIZE 567M REUSE AUTOEXTEND ON NEXT 640K MAXSIZE 32767M; >> D:\oracle\temp\verify.sql
echo. >> D:\oracle\temp\verify.sql
echo SET LINESIZE 200 >> D:\oracle\temp\verify.sql
echo SELECT NAME, OPEN_MODE FROM V$DATABASE; >> D:\oracle\temp\verify.sql
echo SELECT TABLESPACE_NAME, STATUS FROM DBA_TABLESPACES ORDER BY 1; >> D:\oracle\temp\verify.sql
echo EXIT; >> D:\oracle\temp\verify.sql

sqlplus / as sysdba @D:\oracle\temp\verify.sql

echo.
echo ============================================================================
echo DR RESTORE COMPLETED SUCCESSFULLY!
echo ============================================================================
echo Database is OPEN and ready
echo.

endlocal
exit /b 0

---

PHASE 7: Weekly Test Procedure (1 hour first time, 30 min ongoing)

Objective: Document weekly test procedure using new cumulative backup strategy

7.1 Test procedure (run on Saturday morning)

# On Linux workstation or any machine with SSH to Proxmox

# Step 1: Verify latest backups on host (5 min)
ssh root@10.0.20.202 "ls -lth /mnt/pve/oracle-backups/ROA/autobackup/*.bkp | head -10"

# Expected to see:
# - FULL backup from this morning (02:30)
# - CUMULATIVE from yesterday 18:00
# - CUMULATIVE from yesterday 13:00
# - Older files...

# Step 2: Start DR VM (2 min)
ssh root@10.0.20.202 "qm start 109"

# Wait for Windows boot
sleep 180

# Verify VM is up
ping -c 3 10.0.20.37

# Step 3: Verify mount point in VM (2 min)
ssh -p 22122 romfast@10.0.20.37 "Get-ChildItem E:\oracle-backups\ROA\autobackup\*.bkp | Measure-Object"

# Should show ~10-15 backup files

# Step 4: Run restore (15-20 min)
ssh -p 22122 romfast@10.0.20.37 "D:\oracle\scripts\rman_restore_cumulative.cmd"

# Monitor restore progress
ssh -p 22122 romfast@10.0.20.37 "Get-Content D:\oracle\logs\restore_cumulative_*.log -Wait"

# Step 5: Verify database (5 min)
ssh -p 22122 romfast@10.0.20.37 "cmd /c 'set ORACLE_HOME=C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home&& set ORACLE_SID=ROA&& set PATH=%ORACLE_HOME%\bin;%PATH%&& sqlplus -s / as sysdba @D:\oracle\scripts\verify_restore.sql'"

# Step 6: Shutdown VM (2 min)
ssh -p 22122 romfast@10.0.20.37 "shutdown /s /t 60"

# Or force from Proxmox:
ssh root@10.0.20.202 "qm shutdown 109"

# Verify VM stopped
ssh root@10.0.20.202 "qm status 109"

7.2 Create automated test script

#!/bin/bash
# File: /root/scripts/test_oracle_dr.sh
# Run on Linux workstation or Proxmox host

LOG_FILE="/root/scripts/logs/dr_test_$(date +%Y%m%d_%H%M%S).log"
PVEHOST="10.0.20.202"
DRVM="10.0.20.37"
DRVM_PORT="22122"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

log "==================================================================="
log "Oracle DR Weekly Test - Started"
log "==================================================================="

# Step 1: Check backups on host
log "Step 1: Verifying backups on Proxmox host..."
ssh root@$PVEHOST "ls -lh /mnt/pve/oracle-backups/ROA/autobackup/*.bkp | wc -l" | tee -a "$LOG_FILE"

# Step 2: Start DR VM
log "Step 2: Starting DR VM 109..."
ssh root@$PVEHOST "qm start 109"
sleep 180

# Step 3: Verify mount
log "Step 3: Verifying mount point in VM..."
ssh -p $DRVM_PORT romfast@$DRVM "powershell -Command 'Get-ChildItem E:\oracle-backups\ROA\autobackup\*.bkp | Measure-Object'" | tee -a "$LOG_FILE"

# Step 4: Run restore
log "Step 4: Running RMAN restore (this will take 15-20 minutes)..."
ssh -p $DRVM_PORT romfast@$DRVM "D:\oracle\scripts\rman_restore_cumulative.cmd" | tee -a "$LOG_FILE"

if [ $? -eq 0 ]; then
    log "Restore completed successfully"
else
    log "ERROR: Restore failed"
    exit 1
fi

# Step 5: Verify database
log "Step 5: Verifying database..."
ssh -p $DRVM_PORT romfast@$DRVM "cmd /c 'set ORACLE_HOME=C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home&& sqlplus -s / as sysdba @D:\oracle\scripts\verify_restore.sql'" | tee -a "$LOG_FILE"

# Step 6: Shutdown VM
log "Step 6: Shutting down DR VM..."
ssh root@$PVEHOST "qm shutdown 109"
sleep 60

log "==================================================================="
log "Oracle DR Weekly Test - Completed Successfully"
log "==================================================================="

---

📊 EXPECTED RESULTS

Backup Schedule (after implementation)

Time	Type	Size	Retention	Transfer to
02:30	Level 0 FULL	6-7 GB	2 days	Proxmox host
13:00	Level 1 CUMULATIVE	150-300 MB	2 days	Proxmox host
18:00	Level 1 CUMULATIVE	200-400 MB	2 days	Proxmox host

RPO Analysis

Disaster Time	Backup Used	Data Loss
03:00-13:00	FULL (02:30)	Max 10.5 hours
13:00-18:00	FULL + CUMULATIVE (13:00)	Max 5 hours
18:00-02:30	FULL + CUMULATIVE (18:00)	Max 8.5 hours
Average RPO		~4-5 hours

Storage Requirements

• Proxmox host: ~15 GB (2 days × 7.5 GB/day)
• VM 109 disk: 500 GB (unchanged, backups not stored in VM)
• Daily transfer: ~7.5 GB (FULL + 2× CUMULATIVE)

RTO (unchanged)

• Start VM: 2 minutes
• Restore FULL + CUMULATIVE: 12-15 minutes
• Verify & open: 1 minute
• Total: ~15-18 minutes

---

🚨 ROLLBACK PLAN

If any issues during implementation:

Rollback Step 1: Restaurează scripturile originale

# On PRIMARY
cd D:\rman_backup
Copy-Item rman_backup_incremental_ORIGINAL.txt rman_backup_incremental.txt -Force
Copy-Item transfer_incremental_ORIGINAL.ps1 transfer_incremental.ps1 -Force
Copy-Item transfer_to_dr_ORIGINAL.ps1 transfer_to_dr.ps1 -Force

# Verifică că s-au restaurat
Get-Content rman_backup_incremental.txt | Select-String "CUMULATIVE"
# Nu ar trebui să găsească nimic dacă restaurarea a reușit

Rollback Step 2: Restaurează task-urile originale

# Șterge task-ul nou de la 18:00
Unregister-ScheduledTask -TaskName "Oracle RMAN Incremental Backup 1800" -Confirm:$false

# Restaurează task-ul de la 13:00 înapoi la 14:00
$taskName = "Oracle RMAN Incremental Backup"  # Numele task-ului tău
$trigger = New-ScheduledTaskTrigger -Daily -At "14:00"
Set-ScheduledTask -TaskName $taskName -Trigger $trigger

# SAU restaurează din backup XML
Register-ScheduledTask -Xml (Get-Content "D:\rman_backup\backup_tasks\Oracle RMAN Incremental Backup.xml") -Force

---

✅ VALIDATION CHECKLIST

After completing implementation:

• Proxmox host directory created: /mnt/pve/oracle-backups/ROA/autobackup
• NFS server installed and configured on Proxmox
• NFS export configured for VM 109 (10.0.20.37)
• NFS Client enabled in Windows VM 109
• F:\ mount point configured and tested (NFS mount working)
• PowerShell mount script created (C:\Scripts\mount-nfs.ps1)
• Scheduled task "Mount NFS F" created for auto-mount at startup
• F:\ drive persists after VM reboot
• RMAN script modified to CUMULATIVE (keyword added) - Already has CUMULATIVE
• Transfer scripts updated to send to Proxmox host
• SSH key for Proxmox host created and tested
• Scheduled task created for 13:00 CUMULATIVE backup on PRIMARY
• Scheduled task created for 18:00 CUMULATIVE backup on PRIMARY
• Existing 02:30 FULL task updated to use new transfer script
• Manual test of CUMULATIVE backup successful
• Manual test of backup transfer to host successful
• DR restore script updated to use F:\ mount
• Full end-to-end restore test successful
• Weekly test script created and tested
• Documentation updated (STATUS and IMPLEMENTATION_PLAN docs)

---

📞 NEXT SESSION HANDOFF

Status: 🟡 PHASES 1-3-5 COMPLETED - Continue with Phases 4, 6, 7 Estimated Remaining Time: 1.5-2 hours Recommended Schedule: Saturday morning (low activity time)

Context for next session:

1. Primary server: 10.0.20.36 (Windows, Oracle 19c, database ROA)
2. DR VM: 109 on pveelite (10.0.20.37, F:\ NFS mount configured and working ✅)
3. Proxmox host: pveelite (10.0.20.202, NFS server running ✅)
4. Goal: Complete scheduled tasks on PRIMARY + update restore script + test end-to-end
5. Completed: Storage setup, NFS mount, transfer scripts to Proxmox

What's DONE:

• ✅ Proxmox host storage (/mnt/pve/oracle-backups/ROA/autobackup)
• ✅ NFS server on Proxmox with export for VM 109
• ✅ NFS Client in Windows VM 109
• ✅ F:\ drive auto-mounts at startup via scheduled task
• ✅ Transfer scripts modified to send to Proxmox host
• ✅ RMAN script already has CUMULATIVE keyword

Next steps to complete:

# Phase 4 - Update scheduled tasks on PRIMARY (45 min)
# 1. Setup SSH key from PRIMARY to Proxmox
# 2. Test transfer scripts
# 3. Create/modify scheduled tasks for 13:00 and 18:00

# Phase 6 - Update DR restore script (30 min)
# Modify restore script to use F:\ instead of local path

# Phase 7 - End-to-end test (30 min)
# Full test: backup → transfer → mount → restore

IMPORTANT - Backup manual înainte de modificări: Fă backup MANUAL la fișierele pe care le vei modifica:

# Pe PRIMARY, copiază fișierele EXISTENTE înainte de modificare:
cd D:\rman_backup
Copy-Item rman_backup_incremental.txt rman_backup_incremental_ORIGINAL.txt
Copy-Item transfer_incremental.ps1 transfer_incremental_ORIGINAL.ps1
Copy-Item transfer_to_dr.ps1 transfer_to_dr_ORIGINAL.ps1

# Exportă task-urile
Get-ScheduledTask | Where-Object {$_.TaskName -like "*Oracle*"} | ForEach-Object {
    Export-ScheduledTask -TaskName $_.TaskName | Out-File "D:\rman_backup\backup_tasks\$($_.TaskName).xml"
}

Dacă ceva nu merge, restaurezi din aceste copii!

---

Generated: 2025-10-09 Version: 1.0 Author: Claude Code (Sonnet 4.5) Status: ✅ PLAN COMPLETE - Ready for next session implementation