Oracle DR: Complete cleanup and restore scripts with Proxmox integration
- Remove outdated planning documents and implementation guides - Update README with comprehensive DR procedures and monitoring - Enhance rman_restore_from_zero.cmd with SPFILE creation and auto-start - Add Proxmox monitoring and weekly test scripts - Archive old implementation documentation Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
Marius
File diff suppressed because it is too large
Load Diff
356
oracle/standby-server-scripts/archive/DR_VM_MIGRATION_GUIDE.md
Normal file
356
oracle/standby-server-scripts/archive/DR_VM_MIGRATION_GUIDE.md
Normal file
@@ -0,0 +1,356 @@
|
||||
# Oracle DR VM - Migration Between Proxmox Nodes
|
||||
|
||||
**Purpose:** How to migrate VM 109 between Proxmox nodes while maintaining backup access
|
||||
**Scenario:** Move VM from pveelite (10.0.20.202) to pvemini (10.0.20.201) or vice versa
|
||||
|
||||
---
|
||||
|
||||
## 📋 OVERVIEW
|
||||
|
||||
**Current Setup:**
|
||||
- VM 109 runs on pveelite (10.0.20.202)
|
||||
- Backups stored on pveelite: `/mnt/pve/oracle-backups`
|
||||
- VM has mount point: `qm set 109 -mp0 /mnt/pve/oracle-backups`
|
||||
- Mount appears in Windows as **F:\** (E:\ already used)
|
||||
|
||||
**Challenge:**
|
||||
- Mount points are **node-local** - path `/mnt/pve/oracle-backups` exists only on pveelite
|
||||
- If you migrate VM to pvemini, mount point breaks
|
||||
|
||||
**Solution:**
|
||||
- Create same directory structure on destination node
|
||||
- Sync backups between nodes
|
||||
- Mount point works identically on new node
|
||||
|
||||
---
|
||||
|
||||
## 🔄 MIGRATION PROCEDURE
|
||||
|
||||
### PRE-MIGRATION CHECKLIST
|
||||
|
||||
- [ ] VM 109 is powered OFF
|
||||
- [ ] You have root SSH access to both Proxmox nodes
|
||||
- [ ] You know which node you're migrating TO
|
||||
- [ ] Backups are current (check timestamp)
|
||||
|
||||
---
|
||||
|
||||
### STEP 1: Prepare Destination Node (pvemini)
|
||||
|
||||
**On pvemini (10.0.20.201):**
|
||||
|
||||
```bash
|
||||
ssh root@10.0.20.201
|
||||
|
||||
# Create identical directory structure
|
||||
mkdir -p /mnt/pve/oracle-backups/ROA/autobackup
|
||||
chmod 755 /mnt/pve/oracle-backups
|
||||
chmod 755 /mnt/pve/oracle-backups/ROA
|
||||
chmod 755 /mnt/pve/oracle-backups/ROA/autobackup
|
||||
|
||||
# Verify structure
|
||||
ls -la /mnt/pve/oracle-backups/ROA/autobackup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 2: Sync Backups from Source to Destination
|
||||
|
||||
**Option A: Full Sync (first time migration)**
|
||||
|
||||
```bash
|
||||
# On pvemini, sync all backups from pveelite
|
||||
rsync -avz --progress \
|
||||
root@10.0.20.202:/mnt/pve/oracle-backups/ \
|
||||
/mnt/pve/oracle-backups/
|
||||
|
||||
# This copies all backup files (~15 GB, takes 2-3 minutes on 1Gbps network)
|
||||
```
|
||||
|
||||
**Option B: Incremental Sync (if you already synced before)**
|
||||
|
||||
```bash
|
||||
# On pvemini, sync only new/changed files
|
||||
rsync -avz --progress --update \
|
||||
root@10.0.20.202:/mnt/pve/oracle-backups/ \
|
||||
/mnt/pve/oracle-backups/
|
||||
|
||||
# Much faster - only copies new backups
|
||||
```
|
||||
|
||||
**Verify sync:**
|
||||
```bash
|
||||
# Check file count matches
|
||||
ssh root@10.0.20.202 "ls /mnt/pve/oracle-backups/ROA/autobackup/*.bkp | wc -l"
|
||||
ls /mnt/pve/oracle-backups/ROA/autobackup/*.bkp | wc -l
|
||||
|
||||
# Should be same number
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 3: Migrate VM via Proxmox
|
||||
|
||||
**Option A: Online Migration (VM stays running)**
|
||||
|
||||
```bash
|
||||
# From Proxmox CLI on source node (pveelite):
|
||||
qm migrate 109 pvemini --online
|
||||
|
||||
# This uses live migration - VM doesn't stop
|
||||
# Takes 5-10 minutes depending on RAM/disk
|
||||
```
|
||||
|
||||
**Option B: Offline Migration (VM must be stopped)**
|
||||
|
||||
```bash
|
||||
# Stop VM first
|
||||
qm stop 109
|
||||
|
||||
# Migrate
|
||||
qm migrate 109 pvemini
|
||||
|
||||
# Faster than online, but requires downtime
|
||||
```
|
||||
|
||||
**Option C: Via Proxmox Web UI**
|
||||
|
||||
```
|
||||
1. Select VM 109 on pveelite
|
||||
2. Click "Migrate"
|
||||
3. Select target node: pvemini
|
||||
4. Choose migration type: online or offline
|
||||
5. Click "Migrate"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 4: Verify Mount Point After Migration
|
||||
|
||||
**After migration completes:**
|
||||
|
||||
```bash
|
||||
# On pvemini, check VM config includes mount point
|
||||
qm config 109 | grep mp0
|
||||
|
||||
# Expected output:
|
||||
# mp0: /mnt/pve/oracle-backups,mp=/mnt/oracle-backups
|
||||
|
||||
# If missing, add it:
|
||||
qm set 109 -mp0 /mnt/pve/oracle-backups,mp=/mnt/oracle-backups
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 5: Start VM and Verify Access
|
||||
|
||||
```bash
|
||||
# Start VM on new node
|
||||
qm start 109
|
||||
|
||||
# Wait for boot
|
||||
sleep 180
|
||||
|
||||
# Check mount in Windows
|
||||
ssh -p 22122 romfast@10.0.20.37 "Get-PSDrive F"
|
||||
|
||||
# Should show F:\ with Used/Free space
|
||||
|
||||
# Verify backup files accessible
|
||||
ssh -p 22122 romfast@10.0.20.37 "Get-ChildItem F:\ROA\autobackup\*.bkp | Measure-Object"
|
||||
|
||||
# Should show backup file count
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### STEP 6: Update PRIMARY Transfer Scripts
|
||||
|
||||
**On PRIMARY (10.0.20.36):**
|
||||
|
||||
Backup transfer scripts need to know which node to send to.
|
||||
|
||||
**Option A: Update scripts to point to new node**
|
||||
|
||||
```powershell
|
||||
# Edit transfer scripts
|
||||
cd D:\rman_backup
|
||||
|
||||
# Find and replace in transfer scripts:
|
||||
# ÎNAINTE:
|
||||
$DRHost = "10.0.20.202" # pveelite
|
||||
|
||||
# DUPĂ:
|
||||
$DRHost = "10.0.20.201" # pvemini
|
||||
```
|
||||
|
||||
**Option B: Use DNS/hostname (RECOMMENDED)**
|
||||
|
||||
```powershell
|
||||
# In transfer scripts, use hostname instead of IP:
|
||||
$DRHost = "pvedr" # DNS name
|
||||
|
||||
# Then update DNS to point to active node:
|
||||
# pvedr → 10.0.20.201 (currently pvemini)
|
||||
# When you migrate back, just update DNS
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 ONGOING SYNC STRATEGY
|
||||
|
||||
### If VM Stays on New Node Long-Term
|
||||
|
||||
**Setup automated sync from PRIMARY → new node:**
|
||||
|
||||
Just update transfer scripts as in Step 6 above. Backups will now go directly to pvemini.
|
||||
|
||||
**Old backups on pveelite:**
|
||||
- Can be deleted after verification
|
||||
- Or kept as additional backup copy (recommended)
|
||||
|
||||
```bash
|
||||
# On pveelite, cleanup old backups after 7 days
|
||||
find /mnt/pve/oracle-backups/ROA/autobackup -name "*.bkp" -mtime +7 -delete
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### If You Migrate VM Back and Forth
|
||||
|
||||
**Scenario:** VM moves between nodes frequently
|
||||
|
||||
**Solution 1: Sync in both directions**
|
||||
|
||||
```bash
|
||||
# Cronjob on pveelite (every 6 hours)
|
||||
0 */6 * * * rsync -az root@10.0.20.201:/mnt/pve/oracle-backups/ /mnt/pve/oracle-backups/
|
||||
|
||||
# Cronjob on pvemini (every 6 hours)
|
||||
0 */6 * * * rsync -az root@10.0.20.202:/mnt/pve/oracle-backups/ /mnt/pve/oracle-backups/
|
||||
```
|
||||
|
||||
**Solution 2: Shared Storage (NFS/CIFS)**
|
||||
|
||||
Use Proxmox shared storage instead of local paths:
|
||||
- Setup NFS server on one node
|
||||
- Both nodes mount same NFS share
|
||||
- `/mnt/pve/oracle-backups` points to shared storage
|
||||
- VM migration doesn't require backup sync
|
||||
|
||||
---
|
||||
|
||||
## 📊 MIGRATION CHECKLIST
|
||||
|
||||
### Before Migration:
|
||||
- [ ] VM 109 is stopped (or prepared for online migration)
|
||||
- [ ] Destination node has directory: `/mnt/pve/oracle-backups/ROA/autobackup`
|
||||
- [ ] Backups synced to destination node (rsync completed)
|
||||
- [ ] You have tested restore recently (weekly test passed)
|
||||
|
||||
### During Migration:
|
||||
- [ ] VM migration initiated (online or offline)
|
||||
- [ ] Migration progress monitored (no errors)
|
||||
- [ ] Migration completed successfully
|
||||
|
||||
### After Migration:
|
||||
- [ ] VM 109 shows as running on new node
|
||||
- [ ] Mount point configured: `qm config 109 | grep mp0`
|
||||
- [ ] VM started successfully
|
||||
- [ ] F:\ drive accessible in Windows: `Get-PSDrive F`
|
||||
- [ ] Backup files visible: `Get-ChildItem F:\ROA\autobackup\*.bkp`
|
||||
- [ ] PRIMARY transfer scripts updated (point to new node IP)
|
||||
- [ ] Test restore completed successfully
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ TROUBLESHOOTING
|
||||
|
||||
### Mount Point Not Visible in VM After Migration
|
||||
|
||||
**Symptom:** F:\ drive missing in Windows after migration
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# On new node, verify mount point config
|
||||
qm config 109 | grep mp0
|
||||
|
||||
# If missing, add it
|
||||
qm set 109 -mp0 /mnt/pve/oracle-backups,mp=/mnt/oracle-backups
|
||||
|
||||
# Restart VM
|
||||
qm stop 109
|
||||
qm start 109
|
||||
```
|
||||
|
||||
### Backup Files Not Accessible
|
||||
|
||||
**Symptom:** F:\ exists but shows as empty
|
||||
|
||||
**Cause:** Backups not synced to new node
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Re-sync backups from old node
|
||||
rsync -avz root@10.0.20.202:/mnt/pve/oracle-backups/ /mnt/pve/oracle-backups/
|
||||
|
||||
# Verify files exist
|
||||
ls -lh /mnt/pve/oracle-backups/ROA/autobackup/*.bkp
|
||||
```
|
||||
|
||||
### PRIMARY Still Sending to Old Node
|
||||
|
||||
**Symptom:** New backups not appearing on new node
|
||||
|
||||
**Cause:** Transfer scripts still point to old node IP
|
||||
|
||||
**Solution:**
|
||||
Update `$DRHost` in transfer scripts on PRIMARY (see Step 6)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 MIGRATION TIMELINE
|
||||
|
||||
| Task | Duration | Downtime |
|
||||
|------|----------|----------|
|
||||
| Prepare destination node | 5 min | None |
|
||||
| Sync backups (full, ~15GB) | 3 min | None |
|
||||
| Migrate VM (offline) | 5 min | **5 min** |
|
||||
| Verify and start VM | 3 min | **3 min** |
|
||||
| Update PRIMARY scripts | 2 min | None |
|
||||
| **Total** | **18 min** | **8 min** |
|
||||
|
||||
**With online migration:** 0 minutes downtime (VM keeps running during migration)
|
||||
|
||||
---
|
||||
|
||||
## 📞 QUICK REFERENCE
|
||||
|
||||
**Current Setup:**
|
||||
- Source node: pveelite (10.0.20.202)
|
||||
- Destination node: pvemini (10.0.20.201)
|
||||
- VM: 109 (oracle-dr-windows)
|
||||
- Backup path: `/mnt/pve/oracle-backups`
|
||||
- Windows mount: F:\ (not E:\ - already used)
|
||||
|
||||
**Key Commands:**
|
||||
```bash
|
||||
# Sync backups
|
||||
rsync -avz root@SOURCE:/mnt/pve/oracle-backups/ /mnt/pve/oracle-backups/
|
||||
|
||||
# Migrate VM
|
||||
qm migrate 109 DESTINATION --online
|
||||
|
||||
# Check mount
|
||||
qm config 109 | grep mp0
|
||||
|
||||
# Add mount if missing
|
||||
qm set 109 -mp0 /mnt/pve/oracle-backups,mp=/mnt/oracle-backups
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Generated:** 2025-10-09
|
||||
**Version:** 1.0
|
||||
**Status:** Ready for use
|
||||
**See Also:** DR_UPGRADE_TO_CUMULATIVE_PLAN.md
|
||||
@@ -0,0 +1,963 @@
|
||||
# Oracle DR - Windows VM Implementation Plan
|
||||
**Generated:** 2025-10-08
|
||||
**Objective:** Replace Linux LXC DR with Windows VM for same-platform RMAN restore
|
||||
**Target:** Windows VM in Proxmox, IP 10.0.20.37, Oracle 19c SE2
|
||||
|
||||
---
|
||||
|
||||
## 📋 PRE-IMPLEMENTATION CHECKLIST
|
||||
|
||||
### Current Infrastructure (IMPLEMENTED ✅)
|
||||
- ✅ PRIMARY: Windows Server, Oracle 19c SE2, IP: 10.0.20.36, SSH port 22122
|
||||
- ✅ Database: ROA, DBID: 1363569330
|
||||
- ✅ RMAN backups: FULL daily (02:30 AM)
|
||||
- ✅ DIFFERENTIAL INCREMENTAL (14:00) - NOT USED (causes UNDO corruption on restore)
|
||||
- ✅ Transfer scripts: PowerShell scripts transferring to VM 109 (Windows)
|
||||
- ✅ Backup size: ~6-7GB compressed (from 23GB), retention 2 days
|
||||
- ✅ DR target: Windows VM 109 (10.0.20.37) on pveelite - **OPERATIONAL**
|
||||
|
||||
### Planned Upgrade (see DR_UPGRADE_TO_CUMULATIVE_PLAN.md)
|
||||
- 🔄 Convert DIFFERENTIAL → **CUMULATIVE** incremental backups
|
||||
- 🔄 Add second daily incremental (13:00 + 18:00 vs current 14:00 only)
|
||||
- 🔄 Store backups on Proxmox host (pveelite), mounted in VM when needed
|
||||
- 🔄 Target RPO: **3-4 hours** (vs current 24 hours)
|
||||
|
||||
### What We'll Build
|
||||
- 🎯 Windows VM in Proxmox (replaces LXC 109)
|
||||
- 🎯 IP: 10.0.20.37 (same as current LXC)
|
||||
- 🎯 Oracle 19c SE2 installed (empty database template)
|
||||
- 🎯 OpenSSH Server for passwordless transfer
|
||||
- 🎯 RMAN restore scripts (automated DR recovery)
|
||||
- 🎯 Zero daily resource consumption (VM powered off when not needed)
|
||||
|
||||
### Resource Requirements
|
||||
- RAM: 4-6 GB (allocated, but VM runs only during DR events)
|
||||
- Disk: 100 GB (OS + Oracle + backup storage)
|
||||
- CPU: 2-4 vCPU
|
||||
- Network: Access to 10.0.20.0/24
|
||||
|
||||
---
|
||||
|
||||
## 🚀 PHASE 1: CREATE WINDOWS VM IN PROXMOX (30 minutes)
|
||||
|
||||
### Step 1.1: Download Windows 11 ISO
|
||||
```bash
|
||||
# On Proxmox host or download station
|
||||
cd /var/lib/vz/template/iso
|
||||
|
||||
# Option A: Download Windows 11 from Microsoft
|
||||
wget -O Win11_EnglishInternational_x64v1.iso \
|
||||
"https://software-download.microsoft.com/download/pr/..."
|
||||
|
||||
# Option B: Upload existing ISO via Proxmox web UI
|
||||
# Datacenter → Storage → ISO Images → Upload
|
||||
```
|
||||
|
||||
### Step 1.2: Create VM in Proxmox Web UI
|
||||
```
|
||||
Proxmox Web UI → Create VM
|
||||
|
||||
General:
|
||||
- VM ID: 109 (same as LXC number for consistency)
|
||||
- Name: oracle-dr-windows
|
||||
- Start at boot: NO (VM stays off until DR event)
|
||||
|
||||
OS:
|
||||
- ISO: Win11_EnglishInternational_x64v1.iso
|
||||
- Type: Microsoft Windows
|
||||
- Version: 11/2022
|
||||
|
||||
System:
|
||||
- Machine: q35
|
||||
- BIOS: OVMF (UEFI)
|
||||
- Add TPM: YES (for Windows 11)
|
||||
- SCSI Controller: VirtIO SCSI
|
||||
|
||||
Disks:
|
||||
- Bus/Device: SCSI 0
|
||||
- Storage: local-lvm (or your storage)
|
||||
- Size: 100 GB
|
||||
- Cache: Write back
|
||||
- Discard: YES
|
||||
- IO thread: YES
|
||||
|
||||
CPU:
|
||||
- Cores: 4
|
||||
- Type: host
|
||||
|
||||
Memory:
|
||||
- RAM: 6144 MB (6 GB)
|
||||
- Ballooning: NO
|
||||
|
||||
Network:
|
||||
- Bridge: vmbr0
|
||||
- Model: VirtIO
|
||||
- Firewall: NO
|
||||
```
|
||||
|
||||
### Step 1.3: Install Windows 11
|
||||
```
|
||||
1. Start VM → Open Console (noVNC)
|
||||
2. Boot from ISO
|
||||
3. Windows Setup:
|
||||
- Language: English
|
||||
- Install Now
|
||||
- Windows 11 Pro (or your edition)
|
||||
- Custom Install
|
||||
- Load driver: Browse → virtio-win-0.1.x (if needed for disk detection)
|
||||
- Select disk → Format → Next
|
||||
|
||||
4. Initial Setup:
|
||||
- Computer name: ORACLE-DR
|
||||
- Local account: Administrator / <strong-password>
|
||||
- Disable all telemetry/tracking options
|
||||
|
||||
5. First boot:
|
||||
- Disable Windows Defender real-time protection (for Oracle performance)
|
||||
- Disable Windows Update automatic restart
|
||||
- Install VirtIO drivers (guest tools)
|
||||
```
|
||||
|
||||
### Step 1.4: Configure Network (Static IP)
|
||||
```powershell
|
||||
# In Windows VM, run PowerShell as Administrator
|
||||
|
||||
# Set static IP 10.0.20.37
|
||||
New-NetIPAddress -InterfaceAlias "Ethernet" -IPAddress 10.0.20.37 -PrefixLength 24 -DefaultGateway 10.0.20.1
|
||||
|
||||
# Set DNS
|
||||
Set-DnsClientServerAddress -InterfaceAlias "Ethernet" -ServerAddresses ("10.0.20.1","8.8.8.8")
|
||||
|
||||
# Verify
|
||||
Get-NetIPAddress | Where-Object {$_.IPAddress -eq "10.0.20.37"}
|
||||
Test-Connection 10.0.20.36 -Count 2
|
||||
```
|
||||
|
||||
### Step 1.5: Windows Initial Configuration
|
||||
```powershell
|
||||
# Run as Administrator
|
||||
|
||||
# Enable Remote Desktop (optional, for management)
|
||||
Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -Name "fDenyTSConnections" -Value 0
|
||||
Enable-NetFirewallRule -DisplayGroup "Remote Desktop"
|
||||
|
||||
# Disable Windows Firewall for private network (or configure rules)
|
||||
Set-NetFirewallProfile -Profile Domain,Public,Private -Enabled False
|
||||
|
||||
# Set timezone
|
||||
Set-TimeZone -Id "GTB Standard Time" # Romania timezone
|
||||
|
||||
# Disable hibernation (saves disk space)
|
||||
powercfg /hibernate off
|
||||
|
||||
# Create directories for Oracle
|
||||
New-Item -ItemType Directory -Path "D:\oracle" -Force
|
||||
New-Item -ItemType Directory -Path "D:\oracle\backups" -Force
|
||||
New-Item -ItemType Directory -Path "D:\oracle\oradata" -Force
|
||||
New-Item -ItemType Directory -Path "D:\oracle\fra" -Force
|
||||
```
|
||||
|
||||
**✅ PHASE 1 COMPLETE:** Windows VM created, network configured, ready for Oracle installation
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ PHASE 2: INSTALL ORACLE 19c (60-90 minutes)
|
||||
|
||||
### Step 2.1: Download Oracle 19c
|
||||
```
|
||||
On developer machine or PRIMARY:
|
||||
|
||||
1. Go to: https://www.oracle.com/database/technologies/oracle19c-windows-downloads.html
|
||||
2. Download: WINDOWS.X64_193000_db_home.zip (3.0 GB)
|
||||
3. Transfer to VM:
|
||||
- Option A: Shared folder via Proxmox
|
||||
- Option B: HTTP file server
|
||||
- Option C: Direct download in VM
|
||||
```
|
||||
|
||||
### Step 2.2: Prepare Installation (in Windows VM)
|
||||
```powershell
|
||||
# Run as Administrator
|
||||
|
||||
# Extract Oracle installation
|
||||
Expand-Archive -Path "C:\Temp\WINDOWS.X64_193000_db_home.zip" -DestinationPath "D:\oracle\product\19c\dbhome_1"
|
||||
|
||||
# Create response file for silent install
|
||||
$responseFile = @"
|
||||
oracle.install.option=INSTALL_DB_SWONLY
|
||||
UNIX_GROUP_NAME=
|
||||
INVENTORY_LOCATION=D:\oracle\oraInventory
|
||||
ORACLE_HOME=D:\oracle\product\19c\dbhome_1
|
||||
ORACLE_BASE=D:\oracle
|
||||
oracle.install.db.InstallEdition=SE2
|
||||
oracle.install.db.OSDBA_GROUP=ORA_DBA
|
||||
oracle.install.db.OSOPER_GROUP=ORA_OPER
|
||||
oracle.install.db.OSBACKUPDBA_GROUP=ORA_BACKUPDBA
|
||||
oracle.install.db.OSDGDBA_GROUP=ORA_DG
|
||||
oracle.install.db.OSKMDBA_GROUP=ORA_KM
|
||||
oracle.install.db.OSRACDBA_GROUP=ORA_RAC
|
||||
DECLINE_SECURITY_UPDATES=true
|
||||
"@
|
||||
|
||||
$responseFile | Out-File -FilePath "D:\oracle\db_install.rsp" -Encoding ASCII
|
||||
```
|
||||
|
||||
### Step 2.3: Silent Installation
|
||||
```powershell
|
||||
# Run as Administrator
|
||||
|
||||
cd D:\oracle\product\19c\dbhome_1
|
||||
|
||||
# Silent install (takes 30-60 minutes)
|
||||
.\setup.exe -silent -responseFile D:\oracle\db_install.rsp -ignorePrereqFailure
|
||||
|
||||
# Wait for completion, check log:
|
||||
# D:\oracle\oraInventory\logs\installActions<timestamp>.log
|
||||
|
||||
# Run root scripts (as Administrator)
|
||||
D:\oracle\product\19c\dbhome_1\root.bat
|
||||
```
|
||||
|
||||
### Step 2.4: Create Listener
|
||||
```powershell
|
||||
# Set environment
|
||||
$env:ORACLE_HOME = "D:\oracle\product\19c\dbhome_1"
|
||||
$env:PATH = "$env:ORACLE_HOME\bin;$env:PATH"
|
||||
|
||||
# Create listener.ora
|
||||
$listenerOra = @"
|
||||
LISTENER =
|
||||
(DESCRIPTION_LIST =
|
||||
(DESCRIPTION =
|
||||
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.20.37)(PORT = 1521))
|
||||
)
|
||||
)
|
||||
"@
|
||||
|
||||
$listenerOra | Out-File -FilePath "D:\oracle\product\19c\dbhome_1\network\admin\listener.ora" -Encoding ASCII
|
||||
|
||||
# Start listener
|
||||
lsnrctl start
|
||||
|
||||
# Configure listener as Windows service (optional)
|
||||
```
|
||||
|
||||
### Step 2.5: Create Empty Database Template (for faster DR restore)
|
||||
```powershell
|
||||
# Create init parameter file
|
||||
$initROA = @"
|
||||
DB_NAME=ROA
|
||||
DB_BLOCK_SIZE=8192
|
||||
COMPATIBLE=19.0.0
|
||||
MEMORY_TARGET=2G
|
||||
PROCESSES=300
|
||||
OPEN_CURSORS=300
|
||||
DB_RECOVERY_FILE_DEST=D:\oracle\fra
|
||||
DB_RECOVERY_FILE_DEST_SIZE=20G
|
||||
CONTROL_FILES=('D:\oracle\oradata\ROA\control01.ctl','D:\oracle\oradata\ROA\control02.ctl')
|
||||
"@
|
||||
|
||||
$initROA | Out-File -FilePath "D:\oracle\product\19c\dbhome_1\database\initROA.ora" -Encoding ASCII
|
||||
|
||||
# Create directory structure
|
||||
New-Item -ItemType Directory -Path "D:\oracle\oradata\ROA" -Force
|
||||
New-Item -ItemType Directory -Path "D:\oracle\fra" -Force
|
||||
|
||||
# Note: We will NOT create the database now
|
||||
# Database will be created via RMAN RESTORE during DR event
|
||||
```
|
||||
|
||||
**✅ PHASE 2 COMPLETE:** Oracle 19c installed, listener configured, ready for SSH setup
|
||||
|
||||
---
|
||||
|
||||
## 🔐 PHASE 3: CONFIGURE SSH FOR AUTOMATED TRANSFERS (20 minutes)
|
||||
|
||||
### Step 3.1: Install OpenSSH Server
|
||||
```powershell
|
||||
# Run as Administrator
|
||||
|
||||
# Install OpenSSH Server
|
||||
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
|
||||
|
||||
# Start and enable service
|
||||
Start-Service sshd
|
||||
Set-Service -Name sshd -StartupType 'Automatic'
|
||||
|
||||
# Confirm firewall rule
|
||||
Get-NetFirewallRule -Name *ssh*
|
||||
|
||||
# Test SSH from developer machine
|
||||
# ssh Administrator@10.0.20.37
|
||||
```
|
||||
|
||||
### Step 3.2: Configure Passwordless SSH (Key-based Authentication)
|
||||
```powershell
|
||||
# On Windows VM, as Administrator
|
||||
|
||||
# Create .ssh directory
|
||||
$sshDir = "$env:ProgramData\ssh"
|
||||
New-Item -ItemType Directory -Path $sshDir -Force
|
||||
|
||||
# Get public key from PRIMARY server
|
||||
# Option A: Copy manually from PRIMARY C:\Users\Administrator\.ssh\id_rsa.pub
|
||||
# Option B: Download via SCP from developer machine
|
||||
|
||||
# For this example, manually copy the content:
|
||||
# From PRIMARY run: Get-Content C:\Users\Administrator\.ssh\id_rsa.pub
|
||||
|
||||
# On DR Windows VM:
|
||||
$publicKey = "<paste-public-key-here>"
|
||||
$publicKey | Out-File -FilePath "$sshDir\administrators_authorized_keys" -Encoding ASCII
|
||||
|
||||
# Set permissions (CRITICAL for SSH to work)
|
||||
icacls "$sshDir\administrators_authorized_keys" /inheritance:r
|
||||
icacls "$sshDir\administrators_authorized_keys" /grant "SYSTEM:(F)"
|
||||
icacls "$sshDir\administrators_authorized_keys" /grant "BUILTIN\Administrators:(F)"
|
||||
|
||||
# Restart SSH service
|
||||
Restart-Service sshd
|
||||
```
|
||||
|
||||
### Step 3.3: Configure SSH for SYSTEM Account (for scheduled tasks)
|
||||
```powershell
|
||||
# Windows scheduled tasks run as SYSTEM, so we need SYSTEM's SSH key
|
||||
|
||||
# Create SYSTEM's .ssh directory
|
||||
$systemSSHDir = "C:\Windows\System32\config\systemprofile\.ssh"
|
||||
New-Item -ItemType Directory -Path $systemSSHDir -Force
|
||||
|
||||
# Copy the same authorized_keys
|
||||
Copy-Item "$env:ProgramData\ssh\administrators_authorized_keys" `
|
||||
-Destination "$systemSSHDir\authorized_keys" -Force
|
||||
|
||||
# Set permissions
|
||||
icacls "$systemSSHDir\authorized_keys" /inheritance:r
|
||||
icacls "$systemSSHDir\authorized_keys" /grant "SYSTEM:(F)"
|
||||
```
|
||||
|
||||
### Step 3.4: Test SSH Connection from PRIMARY
|
||||
```powershell
|
||||
# On PRIMARY (10.0.20.36), test SSH to DR VM
|
||||
|
||||
# Test 1: Manual connection
|
||||
ssh -i C:\Users\Administrator\.ssh\id_rsa Administrator@10.0.20.37 "echo SSH_OK"
|
||||
|
||||
# Test 2: File transfer
|
||||
echo "test content" > C:\Temp\test.txt
|
||||
scp -i C:\Users\Administrator\.ssh\id_rsa C:\Temp\test.txt Administrator@10.0.20.37:D:\oracle\backups\
|
||||
|
||||
# If successful, you should see the file on DR VM
|
||||
```
|
||||
|
||||
**✅ PHASE 3 COMPLETE:** OpenSSH configured, passwordless authentication working
|
||||
|
||||
---
|
||||
|
||||
## 📝 PHASE 4: UPDATE TRANSFER SCRIPTS (15 minutes)
|
||||
|
||||
### Step 4.1: Modify 02_transfer_to_dr.ps1 for Windows Target
|
||||
```powershell
|
||||
# File: D:\rman_backup\02_transfer_to_dr_windows.ps1
|
||||
# Changes needed:
|
||||
|
||||
# OLD (Linux target):
|
||||
# $DRPath = "/opt/oracle/backups/primary"
|
||||
|
||||
# NEW (Windows target):
|
||||
$DRHost = "10.0.20.37"
|
||||
$DRUser = "Administrator" # Changed from "root"
|
||||
$DRPath = "D:/oracle/backups/primary" # Windows path with forward slashes for SCP
|
||||
$SSHKeyPath = "C:\Users\Administrator\.ssh\id_rsa"
|
||||
|
||||
# Update SSH commands to use Windows paths
|
||||
# Example: Directory creation
|
||||
$null = & ssh -n -i $SSHKeyPath "${DRUser}@${DRHost}" `
|
||||
"New-Item -ItemType Directory -Path '$DRPath' -Force" 2>&1
|
||||
|
||||
# Update cleanup command for Windows
|
||||
function Cleanup-OldBackupsOnDR {
|
||||
Write-Log "Cleaning up old backups on DR (keeping last 2 days)..."
|
||||
|
||||
try {
|
||||
$cleanupCmd = @"
|
||||
Get-ChildItem -Path '$DRPath' -Filter '*.BKP' |
|
||||
Where-Object { `$_.LastWriteTime -lt (Get-Date).AddDays(-2) } |
|
||||
Remove-Item -Force
|
||||
"@
|
||||
$result = & ssh -n -i $SSHKeyPath "${DRUser}@${DRHost}" "powershell -Command `"$cleanupCmd`"" 2>&1
|
||||
|
||||
Write-Log "Cleanup completed on DR"
|
||||
} catch {
|
||||
Write-Log "Cleanup warning: $_" "WARNING"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4.2: Create Updated Transfer Scripts
|
||||
```powershell
|
||||
# Save updated versions:
|
||||
# - 02_transfer_to_dr_windows.ps1 (FULL backup transfer)
|
||||
# - 02b_transfer_incremental_to_dr_windows.ps1 (INCREMENTAL transfer)
|
||||
|
||||
# Key changes for Windows:
|
||||
# 1. DRUser = "Administrator" instead of "root"
|
||||
# 2. DRPath = "D:/oracle/backups/primary" (Windows path)
|
||||
# 3. SSH commands use PowerShell instead of Linux commands
|
||||
# 4. Directory check: Test-Path instead of "test -f"
|
||||
# 5. Cleanup: Get-ChildItem instead of find
|
||||
```
|
||||
|
||||
### Step 4.3: Test Transfer Script
|
||||
```powershell
|
||||
# On PRIMARY, test the new script
|
||||
|
||||
# Manual test
|
||||
D:\rman_backup\02_transfer_to_dr_windows.ps1
|
||||
|
||||
# Check log output
|
||||
Get-Content "D:\rman_backup\logs\transfer_$(Get-Date -Format 'yyyyMMdd').log" -Tail 50
|
||||
|
||||
# Verify on DR VM
|
||||
ssh Administrator@10.0.20.37 "Get-ChildItem D:\oracle\backups\primary"
|
||||
```
|
||||
|
||||
**✅ PHASE 4 COMPLETE:** Transfer scripts updated and tested for Windows target
|
||||
|
||||
---
|
||||
|
||||
## 🔄 PHASE 5: CREATE RMAN RESTORE SCRIPT ON DR VM (30 minutes)
|
||||
|
||||
### Step 5.1: Create RMAN Restore Script
|
||||
```powershell
|
||||
# File: D:\oracle\scripts\rman_restore_from_primary.ps1
|
||||
# Run on DR Windows VM
|
||||
|
||||
param(
|
||||
[string]$BackupPath = "D:\oracle\backups\primary",
|
||||
[string]$OracleHome = "D:\oracle\product\19c\dbhome_1",
|
||||
[string]$OracleBase = "D:\oracle",
|
||||
[string]$DataDir = "D:\oracle\oradata\ROA",
|
||||
[string]$FRADir = "D:\oracle\fra",
|
||||
[int]$DBID = 1363569330,
|
||||
[string]$LogFile = "D:\oracle\logs\restore_$(Get-Date -Format 'yyyyMMdd_HHmmss').log"
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
function Write-Log {
|
||||
param([string]$Message, [string]$Level = "INFO")
|
||||
$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
|
||||
$logLine = "[$timestamp] [$Level] $Message"
|
||||
Write-Host $logLine
|
||||
Add-Content -Path $LogFile -Value $logLine -Encoding UTF8
|
||||
}
|
||||
|
||||
try {
|
||||
Write-Log "======================================================================"
|
||||
Write-Log "Oracle DR Restore - Starting"
|
||||
Write-Log "======================================================================"
|
||||
Write-Log "Backup Path: $BackupPath"
|
||||
Write-Log "Oracle Home: $OracleHome"
|
||||
Write-Log "DBID: $DBID"
|
||||
|
||||
# Set environment
|
||||
$env:ORACLE_HOME = $OracleHome
|
||||
$env:ORACLE_SID = "ROA"
|
||||
$env:PATH = "$OracleHome\bin;$env:PATH"
|
||||
|
||||
# Step 1: Cleanup old database files
|
||||
Write-Log "[1/6] Cleaning old database files..."
|
||||
if (Test-Path $DataDir) {
|
||||
Remove-Item "$DataDir\*" -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
if (Test-Path $FRADir) {
|
||||
Remove-Item "$FRADir\*" -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
New-Item -ItemType Directory -Path $DataDir -Force | Out-Null
|
||||
New-Item -ItemType Directory -Path $FRADir -Force | Out-Null
|
||||
|
||||
# Step 2: Startup NOMOUNT
|
||||
Write-Log "[2/6] Starting instance in NOMOUNT mode..."
|
||||
$sqlNomount = @"
|
||||
STARTUP NOMOUNT PFILE='$OracleHome\database\initROA.ora';
|
||||
EXIT;
|
||||
"@
|
||||
$sqlNomount | sqlplus / as sysdba
|
||||
|
||||
# Step 3: RMAN Restore
|
||||
Write-Log "[3/6] Running RMAN RESTORE CONTROLFILE..."
|
||||
|
||||
$rmanScript = @"
|
||||
SET DBID $DBID;
|
||||
|
||||
RUN {
|
||||
ALLOCATE CHANNEL ch1 DEVICE TYPE DISK;
|
||||
|
||||
# Restore controlfile from autobackup
|
||||
SET CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '$BackupPath/%F';
|
||||
RESTORE CONTROLFILE FROM AUTOBACKUP;
|
||||
}
|
||||
|
||||
EXIT;
|
||||
"@
|
||||
|
||||
$rmanScript | rman TARGET /
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "RMAN RESTORE CONTROLFILE failed"
|
||||
}
|
||||
|
||||
# Step 4: Mount database
|
||||
Write-Log "[4/6] Mounting database..."
|
||||
"ALTER DATABASE MOUNT; EXIT;" | sqlplus / as sysdba
|
||||
|
||||
# Step 5: Catalog and restore database
|
||||
Write-Log "[5/6] Cataloging backups and restoring database..."
|
||||
|
||||
$rmanRestore = @"
|
||||
CATALOG START WITH '$BackupPath/' NOPROMPT;
|
||||
|
||||
RUN {
|
||||
SET NEWNAME FOR DATABASE TO '$DataDir\%b';
|
||||
RESTORE DATABASE;
|
||||
SWITCH DATAFILE ALL;
|
||||
RECOVER DATABASE;
|
||||
}
|
||||
|
||||
EXIT;
|
||||
"@
|
||||
|
||||
$rmanRestore | rman TARGET /
|
||||
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "RMAN RESTORE DATABASE failed"
|
||||
}
|
||||
|
||||
# Step 6: Open database RESETLOGS
|
||||
Write-Log "[6/6] Opening database with RESETLOGS..."
|
||||
"ALTER DATABASE OPEN RESETLOGS; EXIT;" | sqlplus / as sysdba
|
||||
|
||||
Write-Log "======================================================================"
|
||||
Write-Log "DR RESTORE COMPLETED SUCCESSFULLY!"
|
||||
Write-Log "======================================================================"
|
||||
Write-Log "Database ROA is now OPEN and ready"
|
||||
|
||||
# Verify
|
||||
Write-Log "Verification:"
|
||||
$verifySQL = @"
|
||||
SELECT name, open_mode, database_role FROM v`$database;
|
||||
EXIT;
|
||||
"@
|
||||
$verifySQL | sqlplus -s / as sysdba
|
||||
|
||||
exit 0
|
||||
|
||||
} catch {
|
||||
Write-Log "CRITICAL ERROR: $($_.Exception.Message)" "ERROR"
|
||||
Write-Log "Stack trace: $($_.ScriptStackTrace)" "ERROR"
|
||||
exit 1
|
||||
}
|
||||
```
|
||||
|
||||
### Step 5.2: Create Quick Test Script
|
||||
```powershell
|
||||
# File: D:\oracle\scripts\test_restore_latest.ps1
|
||||
# Quick test to verify restore works
|
||||
|
||||
$BackupPath = "D:\oracle\backups\primary"
|
||||
$LatestBackup = Get-ChildItem "$BackupPath\*.BKP" |
|
||||
Sort-Object LastWriteTime -Descending |
|
||||
Select-Object -First 1
|
||||
|
||||
Write-Host "Latest backup: $($LatestBackup.Name)"
|
||||
Write-Host "Size: $([math]::Round($LatestBackup.Length / 1GB, 2)) GB"
|
||||
Write-Host "Date: $($LatestBackup.LastWriteTime)"
|
||||
Write-Host ""
|
||||
Write-Host "Ready to test restore? Run:"
|
||||
Write-Host "D:\oracle\scripts\rman_restore_from_primary.ps1"
|
||||
```
|
||||
|
||||
**✅ PHASE 5 COMPLETE:** RMAN restore script created and ready to test
|
||||
|
||||
---
|
||||
|
||||
## 🧪 PHASE 6: TEST DR RESTORE (30 minutes)
|
||||
|
||||
### Step 6.1: Verify Backups Transferred
|
||||
```powershell
|
||||
# On DR Windows VM
|
||||
|
||||
# Check backup files
|
||||
Get-ChildItem D:\oracle\backups\primary\*.BKP |
|
||||
Sort-Object LastWriteTime -Descending |
|
||||
Select-Object Name, @{N='SizeMB';E={[math]::Round($_.Length/1MB,2)}}, LastWriteTime
|
||||
|
||||
# Expected output: 15-20 files (FULL + INCREMENTAL + CONTROLFILE + SPFILE + ARCHIVELOGS)
|
||||
```
|
||||
|
||||
### Step 6.2: Run Test Restore
|
||||
```powershell
|
||||
# IMPORTANT: This will create a live database on DR VM
|
||||
# Make sure PRIMARY is still running (don't confuse them!)
|
||||
|
||||
# Run restore
|
||||
D:\oracle\scripts\rman_restore_from_primary.ps1
|
||||
|
||||
# Monitor progress in log
|
||||
Get-Content "D:\oracle\logs\restore_*.log" -Wait
|
||||
|
||||
# Expected duration: 10-15 minutes
|
||||
```
|
||||
|
||||
### Step 6.3: Verify Database
|
||||
```powershell
|
||||
# Connect to restored database
|
||||
sqlplus sys/romfastsoft@10.0.20.37:1521/ROA as sysdba
|
||||
|
||||
SQL> SELECT name, open_mode FROM v$database;
|
||||
# Expected: ROA, READ WRITE
|
||||
|
||||
SQL> SELECT tablespace_name, status FROM dba_tablespaces;
|
||||
# Expected: SYSTEM, SYSAUX, UNDOTBS, TS_ROA, USERS - all ONLINE
|
||||
|
||||
SQL> SELECT COUNT(*) FROM dba_tables WHERE owner='<your-app-schema>';
|
||||
# Verify application tables restored
|
||||
|
||||
SQL> EXIT;
|
||||
```
|
||||
|
||||
### Step 6.4: Shutdown DR Database (conserve resources)
|
||||
```powershell
|
||||
# After successful test, shutdown database
|
||||
sqlplus / as sysdba
|
||||
|
||||
SQL> SHUTDOWN IMMEDIATE;
|
||||
SQL> EXIT;
|
||||
|
||||
# Stop listener
|
||||
lsnrctl stop
|
||||
|
||||
# Optional: Shutdown Windows VM to conserve resources
|
||||
# (VM will be started only during actual DR events)
|
||||
```
|
||||
|
||||
**✅ PHASE 6 COMPLETE:** DR restore tested and verified working
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ PHASE 7: UPDATE TASK SCHEDULER ON PRIMARY (10 minutes)
|
||||
|
||||
### Step 7.1: Update Scheduled Tasks to Use New Scripts
|
||||
```powershell
|
||||
# On PRIMARY (10.0.20.36)
|
||||
|
||||
# Task 1: FULL Backup + Transfer (already exists, just update transfer script)
|
||||
# Name: "Oracle RMAN Daily Backup + DR Transfer"
|
||||
# Trigger: Daily 02:30 AM
|
||||
# Action 1: Run RMAN backup (unchanged)
|
||||
# Action 2: UPDATE to new script
|
||||
|
||||
# Update task to use new transfer script
|
||||
$action = New-ScheduledTaskAction -Execute "PowerShell.exe" `
|
||||
-Argument "-NoProfile -ExecutionPolicy Bypass -File D:\rman_backup\02_transfer_to_dr_windows.ps1"
|
||||
|
||||
Set-ScheduledTask -TaskName "Oracle RMAN Daily Backup + DR Transfer" -Action $action
|
||||
|
||||
# Task 2: INCREMENTAL Backup + Transfer
|
||||
# Similar update for incremental task
|
||||
```
|
||||
|
||||
### Step 7.2: Test Scheduled Task Manually
|
||||
```powershell
|
||||
# On PRIMARY
|
||||
|
||||
# Run FULL backup + transfer task manually
|
||||
Start-ScheduledTask -TaskName "Oracle RMAN Daily Backup + DR Transfer"
|
||||
|
||||
# Monitor task status
|
||||
Get-ScheduledTask -TaskName "Oracle RMAN Daily Backup + DR Transfer" |
|
||||
Get-ScheduledTaskInfo
|
||||
|
||||
# Check transfer log
|
||||
Get-Content "D:\rman_backup\logs\transfer_$(Get-Date -Format 'yyyyMMdd').log" -Tail 50
|
||||
|
||||
# Verify on DR
|
||||
ssh Administrator@10.0.20.37 "Get-ChildItem D:\oracle\backups\primary -Filter *.BKP | Measure-Object"
|
||||
```
|
||||
|
||||
**✅ PHASE 7 COMPLETE:** Automated backup and transfer configured
|
||||
|
||||
---
|
||||
|
||||
## 📚 PHASE 8: CREATE DR RUNBOOK (15 minutes)
|
||||
|
||||
### Step 8.1: DR Emergency Procedure
|
||||
```markdown
|
||||
# DISASTER RECOVERY PROCEDURE
|
||||
## When PRIMARY Server (10.0.20.36) Fails
|
||||
|
||||
### PRE-REQUISITES
|
||||
- Proxmox access available
|
||||
- DR Windows VM exists (ID 109)
|
||||
- Latest backups transferred (<24h old)
|
||||
|
||||
### DR ACTIVATION STEPS (RTO: 15-20 minutes)
|
||||
|
||||
1. **Start DR Windows VM (2 minutes)**
|
||||
```
|
||||
Proxmox Web UI → VM 109 (oracle-dr-windows) → Start
|
||||
Wait for Windows to boot
|
||||
Verify network: ping 10.0.20.37
|
||||
```
|
||||
|
||||
2. **Verify Backups Present (1 minute)**
|
||||
```powershell
|
||||
# RDP or Console to 10.0.20.37
|
||||
Get-ChildItem D:\oracle\backups\primary\*.BKP |
|
||||
Sort-Object LastWriteTime -Descending |
|
||||
Select-Object -First 10
|
||||
|
||||
# Verify you see today's or yesterday's backups
|
||||
```
|
||||
|
||||
3. **Run RMAN Restore (12-15 minutes)**
|
||||
```powershell
|
||||
# Run restore script
|
||||
D:\oracle\scripts\rman_restore_from_primary.ps1
|
||||
|
||||
# Monitor log in real-time
|
||||
Get-Content D:\oracle\logs\restore_*.log -Wait
|
||||
```
|
||||
|
||||
4. **Verify Database (2 minutes)**
|
||||
```powershell
|
||||
# Connect to database
|
||||
sqlplus sys/romfastsoft@localhost:1521/ROA as sysdba
|
||||
|
||||
SQL> SELECT name, open_mode FROM v$database;
|
||||
SQL> SELECT tablespace_name, status FROM dba_tablespaces;
|
||||
SQL> -- Verify critical application tables
|
||||
SQL> EXIT;
|
||||
```
|
||||
|
||||
5. **Update Network/DNS (5 minutes)**
|
||||
```
|
||||
- Update DNS: roa-db.example.com → 10.0.20.37
|
||||
- OR: Update application connection strings to 10.0.20.37
|
||||
- Test application connectivity
|
||||
```
|
||||
|
||||
6. **Monitor & Notify**
|
||||
```
|
||||
- Monitor database alert log: D:\oracle\diag\rdbms\roa\ROA\trace\alert_ROA.log
|
||||
- Notify team that DR is active
|
||||
- Document incident timeline
|
||||
```
|
||||
|
||||
### RECOVERY BACK TO PRIMARY (When repaired)
|
||||
|
||||
1. Create fresh RMAN backup from DR (now contains latest data)
|
||||
2. Transfer backup to repaired PRIMARY
|
||||
3. Restore on PRIMARY
|
||||
4. Switch DNS/connections back to PRIMARY
|
||||
5. Shutdown DR VM
|
||||
|
||||
### TESTING SCHEDULE
|
||||
- Monthly DR test: Last Sunday of month
|
||||
- Test duration: 30 minutes
|
||||
- Document test results
|
||||
```
|
||||
|
||||
**✅ PHASE 8 COMPLETE:** DR runbook documented
|
||||
|
||||
---
|
||||
|
||||
## 📊 FINAL ARCHITECTURE
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PRODUCTION ENVIRONMENT │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ PRIMARY (10.0.20.36) - Windows Physical Server │
|
||||
│ ├─ Oracle 19c SE2 │
|
||||
│ ├─ Database: ROA │
|
||||
│ ├─ RMAN Backups: │
|
||||
│ │ ├─ FULL: Daily 02:30 AM (~7GB compressed) │
|
||||
│ │ └─ INCREMENTAL: Daily 14:00 (~50MB) │
|
||||
│ └─ Automatic Transfer to DR via SSH/SCP │
|
||||
│ │
|
||||
│ ↓ SSH Transfer │
|
||||
│ ↓ (950 Mbps) │
|
||||
│ ↓ │
|
||||
│ DR (10.0.20.37) - Windows VM in Proxmox (ID 109) │
|
||||
│ ├─ Oracle 19c SE2 (installed, ready) │
|
||||
│ ├─ VM State: POWERED OFF (0 RAM consumption) │
|
||||
│ ├─ Backups: D:\oracle\backups\primary │
|
||||
│ ├─ Storage: 100 GB (OS + Oracle + backups) │
|
||||
│ └─ Restore Script: D:\oracle\scripts\rman_restore... │
|
||||
│ │
|
||||
│ DR ACTIVATION (when needed): │
|
||||
│ ├─ 1. Power ON VM (2 min) │
|
||||
│ ├─ 2. Run restore script (12 min) │
|
||||
│ ├─ 3. Database OPEN (1 min) │
|
||||
│ └─ TOTAL RTO: ~15 minutes │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
|
||||
METRICS (Current Implementation):
|
||||
- RPO: 24 hours (only FULL backup used; incremental causes UNDO corruption)
|
||||
- RTO: 15 minutes
|
||||
- Storage: 500 GB VM + backups on host
|
||||
- Daily resources: ZERO (VM powered off)
|
||||
- DR test: Weekly (planned)
|
||||
|
||||
METRICS (After Upgrade to CUMULATIVE):
|
||||
- RPO: 3-4 hours (FULL + latest CUMULATIVE)
|
||||
- RTO: 15 minutes (unchanged)
|
||||
- Storage: 500 GB VM + ~15 GB on Proxmox host
|
||||
- Daily resources: ZERO (VM powered off)
|
||||
- DR test: Weekly (automated)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ POST-IMPLEMENTATION CHECKLIST
|
||||
|
||||
### Phase 1-8 (Initial Setup) - ✅ COMPLETED 2025-10-09
|
||||
|
||||
- [x] Windows VM created in Proxmox (VM ID 109, IP 10.0.20.37)
|
||||
- [x] Oracle 19c SE2 installed and working
|
||||
- [x] OpenSSH Server configured with passwordless authentication
|
||||
- [x] Transfer scripts updated and tested (FULL backup)
|
||||
- [x] RMAN restore script created on DR VM
|
||||
- [x] DR restore tested successfully (database opens and is usable)
|
||||
- [x] Scheduled tasks on PRIMARY verified
|
||||
- [x] DR procedures documented
|
||||
- [x] VM shutdown after testing (to conserve resources)
|
||||
|
||||
### Phase 9 (Upgrade to CUMULATIVE) - 📋 PLANNED
|
||||
|
||||
**See:** `DR_UPGRADE_TO_CUMULATIVE_PLAN.md` for detailed implementation steps
|
||||
|
||||
- [ ] Proxmox host storage configured (`/mnt/pve/oracle-backups`)
|
||||
- [ ] VM 109 mount point configured (E:\ from host)
|
||||
- [ ] RMAN script updated to CUMULATIVE incremental
|
||||
- [ ] Transfer scripts updated to send to Proxmox host
|
||||
- [ ] SSH key for Proxmox host access configured
|
||||
- [ ] Scheduled task created for 13:00 CUMULATIVE backup
|
||||
- [ ] Scheduled task created for 18:00 CUMULATIVE backup
|
||||
- [ ] Existing 14:00 task removed
|
||||
- [ ] 02:30 FULL task updated to use new transfer script
|
||||
- [ ] DR restore script updated for cumulative backups
|
||||
- [ ] End-to-end restore test with CUMULATIVE successful
|
||||
- [ ] Weekly test script created and scheduled
|
||||
- [ ] Team trained on new backup strategy
|
||||
|
||||
---
|
||||
|
||||
## 🔧 TROUBLESHOOTING GUIDE
|
||||
|
||||
### Issue: SSH Connection Fails
|
||||
```powershell
|
||||
# Check 1: SSH service running?
|
||||
Get-Service sshd
|
||||
|
||||
# Check 2: Firewall blocking?
|
||||
Get-NetFirewallRule -Name *ssh*
|
||||
|
||||
# Check 3: Authorized keys permissions?
|
||||
icacls "C:\ProgramData\ssh\administrators_authorized_keys"
|
||||
|
||||
# Check 4: Test from PRIMARY
|
||||
ssh -v Administrator@10.0.20.37
|
||||
```
|
||||
|
||||
### Issue: RMAN Restore Fails "CONTROLFILE not found"
|
||||
```
|
||||
# This is the cross-platform issue!
|
||||
# Solution: Ensure you're using Windows→Windows (same platform)
|
||||
# Check Oracle version matches: 19c on both sides
|
||||
```
|
||||
|
||||
### Issue: Database Won't Start
|
||||
```powershell
|
||||
# Check alert log
|
||||
Get-Content D:\oracle\diag\rdbms\roa\ROA\trace\alert_ROA.log -Tail 100
|
||||
|
||||
# Check parameter file
|
||||
Get-Content D:\oracle\product\19c\dbhome_1\database\initROA.ora
|
||||
|
||||
# Verify directories exist
|
||||
Test-Path D:\oracle\oradata\ROA
|
||||
Test-Path D:\oracle\fra
|
||||
```
|
||||
|
||||
### Issue: VM Uses Too Much Disk
|
||||
```powershell
|
||||
# Check backup retention
|
||||
Get-ChildItem D:\oracle\backups\primary\*.BKP |
|
||||
Where-Object { $_.LastWriteTime -lt (Get-Date).AddDays(-3) } |
|
||||
Remove-Item -Force
|
||||
|
||||
# Check FRA usage
|
||||
SELECT * FROM V$RECOVERY_FILE_DEST;
|
||||
|
||||
# Cleanup old archives
|
||||
RMAN> DELETE NOPROMPT ARCHIVELOG ALL COMPLETED BEFORE 'SYSDATE-2';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📞 SUPPORT & REFERENCES
|
||||
|
||||
### Oracle Documentation
|
||||
- RMAN Backup and Recovery: https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/
|
||||
- Cross-Platform Migration: https://docs.oracle.com/en/database/oracle/oracle-database/19/spmds/
|
||||
- Windows Installation: https://docs.oracle.com/en/database/oracle/oracle-database/19/ntqrf/
|
||||
|
||||
### Internal Scripts
|
||||
- PRIMARY RMAN backup: D:\rman_backup\rman_backup.txt
|
||||
- Transfer script (FULL): D:\rman_backup\02_transfer_to_dr_windows.ps1
|
||||
- Transfer script (INCREMENTAL): D:\rman_backup\02b_transfer_incremental_to_dr_windows.ps1
|
||||
- DR restore script: D:\oracle\scripts\rman_restore_from_primary.ps1 (on DR VM)
|
||||
|
||||
### Logs Location
|
||||
- PRIMARY transfer logs: D:\rman_backup\logs\
|
||||
- DR restore logs: D:\oracle\logs\
|
||||
- Oracle alert log: D:\oracle\diag\rdbms\roa\ROA\trace\alert_ROA.log
|
||||
|
||||
---
|
||||
|
||||
## 🎯 IMPLEMENTATION TIMELINE
|
||||
|
||||
| Phase | Task | Duration | Responsible |
|
||||
|-------|------|----------|-------------|
|
||||
| 1 | Create Windows VM in Proxmox | 30 min | Infrastructure Admin |
|
||||
| 2 | Install Oracle 19c | 90 min | DBA |
|
||||
| 3 | Configure SSH | 20 min | Infrastructure Admin |
|
||||
| 4 | Update Transfer Scripts | 15 min | DBA |
|
||||
| 5 | Create Restore Script | 30 min | DBA |
|
||||
| 6 | Test DR Restore | 30 min | DBA |
|
||||
| 7 | Update Scheduled Tasks | 10 min | DBA |
|
||||
| 8 | Document DR Runbook | 15 min | DBA |
|
||||
| **TOTAL** | | **~4 hours** | |
|
||||
|
||||
**Note:** This is one-time setup. After completion, daily operations are fully automated with ZERO maintenance overhead.
|
||||
|
||||
---
|
||||
|
||||
**Generated:** 2025-10-08
|
||||
**Last Updated:** 2025-10-09
|
||||
**Version:** 2.0
|
||||
**Status:** ✅ Phase 1-8 COMPLETED | 📋 Phase 9 (CUMULATIVE upgrade) PLANNED
|
||||
**Implementation Status:**
|
||||
- Initial setup (Phases 1-8): ✅ COMPLETED 2025-10-09
|
||||
- RMAN restore tested: ✅ SUCCESSFUL (12-15 minutes RTO)
|
||||
- Current RPO: 24 hours (FULL backup only)
|
||||
- Next: Upgrade to CUMULATIVE incremental for 3-4 hour RPO
|
||||
|
||||
**Next Session:** Implement CUMULATIVE backup strategy
|
||||
**See:** `DR_UPGRADE_TO_CUMULATIVE_PLAN.md` for upgrade plan
|
||||
|
||||
@@ -0,0 +1,789 @@
|
||||
# Oracle DR Windows VM - Implementation Status
|
||||
**Date:** 2025-10-09 04:00 AM
|
||||
**VM:** 109 (oracle-dr-windows)
|
||||
**Location:** Proxmox pveelite (10.0.20.202)
|
||||
**IP:** 10.0.20.37
|
||||
**Purpose:** Replace Linux LXC DR with Windows VM for same-platform RMAN restore
|
||||
|
||||
---
|
||||
|
||||
## ✅ COMPLETED TASKS
|
||||
|
||||
### 1. VM Creation and Network ✅
|
||||
- **VM ID:** 109 on pveelite (10.0.20.202)
|
||||
- **Template source:** Win11-Template (ID 300) from pvemini (10.0.20.201)
|
||||
- **Cloned and migrated:** Successfully migrated from pvemini to pveelite
|
||||
- **Resources configured:**
|
||||
- RAM: 6GB
|
||||
- CPU: 4 cores
|
||||
- Disk: 500GB (local-zfs)
|
||||
- Boot on startup: NO (VM stays off until DR event)
|
||||
- **Network:**
|
||||
- Static IP: 10.0.20.37
|
||||
- Gateway: 10.0.20.1
|
||||
- DNS: 10.0.20.1, 8.8.8.8
|
||||
- Windows Firewall: Disabled
|
||||
- Connectivity: ✅ Verified (ping successful)
|
||||
|
||||
### 2. Windows Configuration ✅
|
||||
- **Computer name:** ORACLE-DR
|
||||
- **Timezone:** GTB Standard Time (Romania)
|
||||
- **Hibernation:** Disabled
|
||||
- **Administrator profile:** Fixed (C:\Users\Administrator)
|
||||
- **Auto-login:** Disabled
|
||||
|
||||
### 3. Users Created ✅
|
||||
| User | Password | Admin | Hidden from Login | Purpose |
|
||||
|------|----------|-------|-------------------|---------|
|
||||
| romfast | Romfast2025! | Yes | Yes | SSH access, backup transfers |
|
||||
| silvia | Silvia2025! | No | Yes | SSH tunnels (2 ports) |
|
||||
| eli | Eli2025! | No | Yes | SSH tunnels (4 ports) |
|
||||
|
||||
### 4. OpenSSH Server Configuration ✅
|
||||
- **Port:** 22122
|
||||
- **Service:** Running, Automatic startup
|
||||
- **Authentication:** ✅ **SSH Key Authentication WORKING**
|
||||
- User key: `mmarius28@gmail.com` (for manual SSH from Linux)
|
||||
- SYSTEM key: `administrator@ROA-CARAPETRU2` (for automated backup transfers from PRIMARY)
|
||||
|
||||
**SSH Config:** `C:\ProgramData\ssh\sshd_config`
|
||||
```
|
||||
Port 22122
|
||||
ListenAddress 0.0.0.0
|
||||
PubkeyAuthentication yes
|
||||
PasswordAuthentication yes
|
||||
AuthorizedKeysFile .ssh/authorized_keys
|
||||
AllowTcpForwarding yes
|
||||
GatewayPorts yes
|
||||
|
||||
Match User romfast
|
||||
PermitOpen localhost:80 localhost:1521 localhost:3000 localhost:3001 localhost:3389 localhost:8006 localhost:8080 localhost:81 localhost:9443 localhost:22
|
||||
|
||||
Match User silvia
|
||||
PermitOpen localhost:80 localhost:1521
|
||||
|
||||
Match User eli
|
||||
PermitOpen localhost:80 localhost:1521 localhost:3000
|
||||
|
||||
Match Group administrators
|
||||
AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys
|
||||
```
|
||||
|
||||
**SSH Keys Configured:**
|
||||
- File: `C:\ProgramData\ssh\administrators_authorized_keys`
|
||||
- Contains 2 keys:
|
||||
1. `ssh-rsa ...mmarius28@gmail.com` (your Linux workstation)
|
||||
2. `ssh-rsa ...administrator@ROA-CARAPETRU2` (PRIMARY SYSTEM user for automated transfers)
|
||||
- Permissions: SYSTEM (Full Control), Administrators (Read)
|
||||
- Status: ✅ Both keys working
|
||||
|
||||
**Fix Script:** `D:\oracle\scripts\fix_ssh_via_service.ps1`
|
||||
- Stops SSH service
|
||||
- Recreates authorized_keys with both keys
|
||||
- Sets correct permissions using `icacls`
|
||||
- Restarts SSH service
|
||||
|
||||
### 5. Oracle 19c Installation ✅
|
||||
- **Status:** ✅ Installed (interactive GUI installation)
|
||||
- **ORACLE_HOME:** `C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home`
|
||||
- **ORACLE_BASE:** `C:\Users\oracle`
|
||||
- **Edition:** Standard Edition 2 (SE2)
|
||||
- **Version:** 19.3.0.0.0
|
||||
- **Installation Type:** Software Only (no database created yet)
|
||||
- **Oracle User:** `oracle` (password: Oracle2025!)
|
||||
|
||||
**Verification:**
|
||||
```powershell
|
||||
$env:ORACLE_HOME = "C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home"
|
||||
$env:PATH = "$env:ORACLE_HOME\bin;$env:PATH"
|
||||
sqlplus -v # Returns: SQL*Plus: Release 19.0.0.0.0 - Production
|
||||
```
|
||||
|
||||
### 6. Oracle Listener Configuration ✅
|
||||
- **Script:** `D:\oracle\scripts\configure_listener_dr.ps1`
|
||||
- **Status:** ✅ Configured and Running
|
||||
- **Port:** 1521
|
||||
- **Service:** OracleOraDB19Home1TNSListener
|
||||
|
||||
**Configuration Files Created:**
|
||||
- `C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home\network\admin\listener.ora`
|
||||
- `C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home\network\admin\tnsnames.ora`
|
||||
- `C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home\network\admin\sqlnet.ora`
|
||||
|
||||
**Listener Status:**
|
||||
```
|
||||
LSNRCTL for 64-bit Windows: Version 19.0.0.0.0 - Production
|
||||
STATUS of the LISTENER
|
||||
Alias LISTENER
|
||||
Version TNSLSNR for 64-bit Windows: Version 19.0.0.0.0 - Production
|
||||
Start Date 09-OCT-2025 03:18:34
|
||||
Listening Endpoints Summary...
|
||||
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.0.20.37)(PORT=1521)))
|
||||
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROC1521ipc)))
|
||||
Services Summary...
|
||||
Service "ROA" has 1 instance(s).
|
||||
Instance "ROA", status UNKNOWN, has 1 handler(s) for this service...
|
||||
```
|
||||
|
||||
### 7. Directory Structure Created ✅
|
||||
```
|
||||
C:\Users\oracle\
|
||||
├── oradata\ROA\ (will be created by RMAN restore)
|
||||
├── recovery_area\ROA\ (FRA - Fast Recovery Area)
|
||||
├── admin\ROA\
|
||||
│ ├── adump\ (audit files)
|
||||
│ ├── dpdump\ (data pump)
|
||||
│ └── pfile\ (initialization files)
|
||||
└── oraInventory\ (Oracle inventory)
|
||||
|
||||
D:\oracle\
|
||||
├── backups\primary\ ✅ (6.32 GB backup files transferred)
|
||||
├── scripts\ ✅ (DR automation scripts)
|
||||
└── logs\ ✅ (restore logs)
|
||||
```
|
||||
|
||||
### 8. Backup Transfer Scripts Updated ✅
|
||||
**Location on PRIMARY:** `D:\rman_backup\`
|
||||
|
||||
**Scripts Updated:**
|
||||
1. **transfer_to_dr.ps1** - Transfer FULL backups
|
||||
2. **transfer_incremental.ps1** - Transfer INCREMENTAL backups
|
||||
|
||||
**Changes Made:**
|
||||
- ✅ DRHost: `10.0.20.37`
|
||||
- ✅ DRPort: `22122` (added)
|
||||
- ✅ DRUser: `romfast` (changed from `root`)
|
||||
- ✅ DRPath: `D:/oracle/backups/primary` (changed from `/opt/oracle/backups/primary`)
|
||||
- ✅ All SSH commands updated with `-p 22122`
|
||||
- ✅ Linux commands replaced with Windows PowerShell equivalents:
|
||||
- `test -f` → `powershell -Command "Test-Path ..."`
|
||||
- `mkdir -p` → `powershell -Command "New-Item -ItemType Directory ..."`
|
||||
- `find ... -delete` → `powershell -Command "Get-ChildItem ... | Remove-Item ..."`
|
||||
|
||||
**Backup Files Transferred:** ✅ **6 files, 6.32 GB total**
|
||||
```
|
||||
D:\oracle\backups\primary\
|
||||
├── O1_MF_NNND0_DAILY_FULL_COMPRESSE_NGFVB4B8_.BKP (4.81 GB) # FULL backup
|
||||
├── O1_MF_ANNNN_DAILY_FULL_COMPRESSE_NGFV7RGN_.BKP (1.51 GB) # FULL backup
|
||||
├── O1_MF_NCNNF_TAG20251009T020551_NGFVLJTG_.BKP (1.14 MB) # Control file
|
||||
├── O1_MF_S_1214013953_NGFVLL29_.BKP (1.14 MB) # SPFILE autobackup
|
||||
├── O1_MF_NNSNF_TAG20251009T020550_NGFVLGOR_.BKP (112 KB)
|
||||
└── O1_MF_ANNNN_DAILY_FULL_COMPRESSE_NGFVLFKN_.BKP (861 KB)
|
||||
```
|
||||
|
||||
**Transfer Log:** `D:\rman_backup\logs\transfer_20251009.log`
|
||||
```
|
||||
[2025-10-09 03:52:13] [SUCCESS] SSH connection successful
|
||||
[2025-10-09 03:52:14] [INFO] Found 6 files, total size: 6.32 GB
|
||||
[2025-10-09 03:57:27] [INFO] Files transferred: 6/6
|
||||
```
|
||||
|
||||
### 9. DR Scripts Created ✅
|
||||
All scripts located in: `/mnt/e/proiecte/ROMFASTSQL/oracle/standby-server-scripts/`
|
||||
|
||||
**Installation Scripts:**
|
||||
1. ✅ `install_oracle19c_dr.ps1` - Oracle 19c installation (software only)
|
||||
2. ✅ `configure_listener_dr.ps1` - Oracle Listener configuration
|
||||
|
||||
**SSH Configuration Scripts:**
|
||||
3. ✅ `fix_ssh_key_auth.ps1` - Initial SSH key setup attempt
|
||||
4. ✅ `fix_ssh_key_auth_simple.cmd` - Simple command-line version
|
||||
5. ✅ `fix_ssh_via_service.ps1` - **WORKING** - Fixes SSH keys by stopping service
|
||||
|
||||
**Backup Transfer Scripts (on PRIMARY):**
|
||||
6. ✅ `transfer_to_dr.ps1` - Full backup transfer (updated for Windows)
|
||||
7. ✅ `transfer_incremental.ps1` - Incremental backup transfer (updated for Windows)
|
||||
8. ✅ `transfer_to_dr_windows.ps1` - Reference implementation
|
||||
|
||||
**Restore Script:**
|
||||
9. ✅ `rman_restore_from_primary.ps1` - RMAN restore script (ready to test)
|
||||
|
||||
**Helper Scripts:**
|
||||
10. ✅ `copy_system_ssh_key.ps1` - Extract SYSTEM user SSH key from PRIMARY
|
||||
11. ✅ `add_system_key_dr.ps1` - Add SYSTEM key to DR VM
|
||||
|
||||
---
|
||||
|
||||
## ✅ RMAN RESTORE COMPLETED - 2025-10-09 17:40
|
||||
|
||||
### 10. RMAN Restore End-to-End Test ✅ **COMPLETED**
|
||||
|
||||
**Final Status:** ✅ **DATABASE SUCCESSFULLY RESTORED AND OPEN**
|
||||
- Database: ROA
|
||||
- Mode: READ WRITE
|
||||
- Instance: OPEN
|
||||
- Tablespaces: 6 (all ONLINE)
|
||||
- Datafiles: 5
|
||||
- Application Owners: 69
|
||||
- Total Application Tables: 45,000+
|
||||
|
||||
**Session Duration:** ~5 hours (including troubleshooting)
|
||||
**Actual Restore Time:** ~15-20 minutes (datafiles + recovery)
|
||||
**Total Data Restored:** 6.32 GB compressed → ~15 GB uncompressed
|
||||
|
||||
---
|
||||
|
||||
## 🔧 CRITICAL ISSUES ENCOUNTERED & RESOLUTIONS
|
||||
|
||||
### Issue 1: Incremental Backup Corruption ⚠️ → ✅ RESOLVED
|
||||
**Problem:** Applying DIFFERENTIAL incremental backup (MIDDAY_INCREMENTAL from 14:00) caused UNDO tablespace corruption
|
||||
- Error: ORA-30012: undo tablespace 'UNDOTBS01' does not exist or of wrong type
|
||||
- Error: ORA-00603: ORACLE server session terminated by fatal error
|
||||
- Database crashed immediately after OPEN RESETLOGS attempt
|
||||
|
||||
**Root Cause:** DIFFERENTIAL incremental backup applied on top of FULL backup created inconsistent UNDO state
|
||||
|
||||
**Initial Workaround:** Restore only FULL backup without applying incremental
|
||||
|
||||
**Permanent Solution:** ✅ **Upgrade to CUMULATIVE incremental backups**
|
||||
- CUMULATIVE backups are independent from Level 0 (no dependency chain)
|
||||
- Each CUMULATIVE contains ALL changes since last Level 0
|
||||
- Eliminates UNDO/SCN mismatch issues
|
||||
- **See:** `DR_UPGRADE_TO_CUMULATIVE_PLAN.md` for implementation plan
|
||||
|
||||
### Issue 2: Control File SCN Mismatch 🔴
|
||||
**Problem:** ORA-01190: control file or data file 1 is from before the last RESETLOGS
|
||||
- Control file autobackup (`O1_MF_S_1214013953_NGFVLL29_.BKP`) created AFTER datafiles backup
|
||||
- SCN in control file was higher than SCN in datafiles
|
||||
- Error: ORA-01152: file 1 was not restored from a sufficiently old backup
|
||||
|
||||
**Root Cause:** Used SPFILE/Controlfile AUTOBACKUP instead of control file from same backup piece as datafiles
|
||||
|
||||
**Resolution:**
|
||||
1. Restore control file from SAME backup as datafiles: `O1_MF_NCNNF_TAG20251009T020551_NGFVLJTG_.BKP`
|
||||
2. This control file has matching SCN with datafiles (both from 02:05:51 backup)
|
||||
|
||||
### Issue 3: ORA-16433 Recovery Loop 🔄
|
||||
**Problem:** ORA-16433: The database or pluggable database must be opened in read/write mode
|
||||
- Occurred during RECOVER DATABASE attempts
|
||||
- Error appeared in both SQL*Plus and RMAN
|
||||
- Recovery session canceled due to errors
|
||||
|
||||
**Root Cause:**
|
||||
- Bug 14744052: Flag set in control file during incomplete RESETLOGS
|
||||
- Using `SET UNTIL SCN 999999999999` in RMAN caused invalid recovery state
|
||||
- Standard Edition limitations with recovery operations
|
||||
|
||||
**Resolution:**
|
||||
1. Remove `SET UNTIL SCN` from RMAN script
|
||||
2. Use `SET UNTIL TIME` with specific backup completion time
|
||||
3. Let RMAN auto-detect and apply only available archive logs
|
||||
4. Incomplete recovery flag properly set by stopping at missing archive log
|
||||
|
||||
### Issue 4: Memory Configuration ⚠️
|
||||
**Problem:** ORA-27104: system-defined limits for shared memory was misconfigured
|
||||
- Initial PFILE had `memory_target=1536M`
|
||||
- VM has 6GB RAM but Windows reserved ~2GB
|
||||
- Database startup failed in NOMOUNT
|
||||
|
||||
**Resolution:**
|
||||
Reduced memory settings in PFILE:
|
||||
```
|
||||
memory_target=1024M
|
||||
memory_max_target=1024M
|
||||
```
|
||||
|
||||
### Issue 5: Backup Location Issues 📁
|
||||
**Initial Setup:** Backups in `D:\oracle\backups\primary` (custom path)
|
||||
- RMAN couldn't auto-detect backups
|
||||
- Had to specify explicit paths for all operations
|
||||
- Control file autobackup search failed
|
||||
|
||||
**Final Solution:**
|
||||
1. Moved all backups to FRA: `C:\Users\oracle\recovery_area\ROA\autobackup`
|
||||
2. Updated PRIMARY transfer scripts to use FRA path
|
||||
3. RMAN now auto-detects all backups via CATALOG command
|
||||
4. Simplified restore procedure significantly
|
||||
|
||||
---
|
||||
|
||||
## 📋 WORKING RMAN RESTORE PROCEDURE
|
||||
|
||||
### Prerequisites ✅ ALL COMPLETE
|
||||
- ✅ Oracle 19c installed on DR VM
|
||||
- ✅ Listener configured and running
|
||||
- ✅ FULL backup transferred from PRIMARY to FRA location
|
||||
- ✅ OracleServiceROA Windows service created
|
||||
- ✅ Backups moved to: `C:\Users\oracle\recovery_area\ROA\autobackup`
|
||||
|
||||
### Step-by-Step Manual Procedure (Tested and Verified)
|
||||
|
||||
**1. Prepare PFILE (Modified for DR)**
|
||||
Location: `C:\Users\oracle\admin\ROA\pfile\initROA.ora`
|
||||
```ini
|
||||
db_name=ROA
|
||||
memory_target=1024M
|
||||
memory_max_target=1024M
|
||||
processes=150
|
||||
undo_management=MANUAL
|
||||
compatible=19.0.0
|
||||
control_files=('C:\Users\oracle\oradata\ROA\control01.ctl', 'C:\Users\oracle\recovery_area\ROA\control02.ctl')
|
||||
db_block_size=8192
|
||||
db_recovery_file_dest=C:\Users\Oracle\recovery_area
|
||||
db_recovery_file_dest_size=10G
|
||||
diagnostic_dest=C:\Users\oracle
|
||||
```
|
||||
|
||||
**2. Shutdown Database (if running)**
|
||||
```cmd
|
||||
set ORACLE_HOME=C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home
|
||||
set ORACLE_SID=ROA
|
||||
set PATH=%ORACLE_HOME%\bin;%PATH%
|
||||
|
||||
sqlplus / as sysdba
|
||||
SHUTDOWN ABORT;
|
||||
EXIT;
|
||||
```
|
||||
|
||||
**3. Startup NOMOUNT**
|
||||
```sql
|
||||
STARTUP NOMOUNT PFILE='C:\Users\oracle\admin\ROA\pfile\initROA.ora';
|
||||
EXIT;
|
||||
```
|
||||
|
||||
**4. Connect to RMAN and Restore Control File**
|
||||
```cmd
|
||||
rman target /
|
||||
|
||||
SET DBID 1363569330;
|
||||
|
||||
RUN {
|
||||
ALLOCATE CHANNEL ch1 DEVICE TYPE DISK;
|
||||
RESTORE CONTROLFILE FROM 'C:/Users/oracle/recovery_area/ROA/autobackup/O1_MF_NCNNF_TAG20251009T020551_NGFVLJTG_.BKP';
|
||||
RELEASE CHANNEL ch1;
|
||||
}
|
||||
|
||||
ALTER DATABASE MOUNT;
|
||||
```
|
||||
|
||||
**5. Catalog Backups in FRA**
|
||||
```rman
|
||||
CATALOG START WITH 'C:/Users/oracle/recovery_area/ROA/autobackup' NOPROMPT;
|
||||
```
|
||||
|
||||
**6. Restore and Recover Database**
|
||||
```rman
|
||||
RUN {
|
||||
ALLOCATE CHANNEL ch1 DEVICE TYPE DISK;
|
||||
ALLOCATE CHANNEL ch2 DEVICE TYPE DISK;
|
||||
SET UNTIL TIME "TO_DATE('09-OCT-2025 02:05:51','DD-MON-YYYY HH24:MI:SS')";
|
||||
RESTORE DATABASE;
|
||||
RECOVER DATABASE;
|
||||
RELEASE CHANNEL ch1;
|
||||
RELEASE CHANNEL ch2;
|
||||
}
|
||||
```
|
||||
|
||||
**7. Open Database with RESETLOGS**
|
||||
```rman
|
||||
ALTER DATABASE OPEN RESETLOGS;
|
||||
EXIT;
|
||||
```
|
||||
|
||||
**8. Create TEMP Tablespace**
|
||||
```sql
|
||||
sqlplus / as sysdba
|
||||
|
||||
ALTER TABLESPACE TEMP ADD TEMPFILE 'C:\Users\oracle\oradata\ROA\temp01.dbf'
|
||||
SIZE 567M REUSE AUTOEXTEND ON NEXT 640K MAXSIZE 32767M;
|
||||
|
||||
EXIT;
|
||||
```
|
||||
|
||||
**9. Verify Database Status**
|
||||
```sql
|
||||
sqlplus / as sysdba
|
||||
|
||||
SELECT NAME, OPEN_MODE, LOG_MODE FROM V$DATABASE;
|
||||
SELECT INSTANCE_NAME, STATUS FROM V$INSTANCE;
|
||||
SELECT TABLESPACE_NAME, STATUS FROM DBA_TABLESPACES ORDER BY TABLESPACE_NAME;
|
||||
SELECT COUNT(*) AS DATAFILE_COUNT FROM DBA_DATA_FILES;
|
||||
|
||||
SELECT OWNER, COUNT(*) AS TABLE_COUNT
|
||||
FROM DBA_TABLES
|
||||
WHERE OWNER NOT IN ('SYS','SYSTEM','OUTLN','MDSYS','CTXSYS','XDB','WMSYS','OLAPSYS',
|
||||
'ORDDATA','ORDSYS','EXFSYS','LBACSYS','DBSNMP','APPQOSSYS','GSMADMIN_INTERNAL')
|
||||
GROUP BY OWNER
|
||||
ORDER BY OWNER;
|
||||
|
||||
EXIT;
|
||||
```
|
||||
|
||||
### Expected Results ✅ VERIFIED
|
||||
|
||||
**Database Status:**
|
||||
```
|
||||
NAME: ROA
|
||||
OPEN_MODE: READ WRITE
|
||||
LOG_MODE: ARCHIVELOG
|
||||
INSTANCE_NAME: ROA
|
||||
STATUS: OPEN
|
||||
```
|
||||
|
||||
**Tablespaces:**
|
||||
```
|
||||
SYSAUX ONLINE
|
||||
SYSTEM ONLINE
|
||||
TEMP ONLINE
|
||||
TS_ROA ONLINE
|
||||
UNDOTBS01 ONLINE
|
||||
USERS ONLINE
|
||||
```
|
||||
|
||||
**Data Verification:**
|
||||
- Datafiles: 5 (excluding TEMP)
|
||||
- Application Owners: 69
|
||||
- Application Tables: 45,000+
|
||||
|
||||
**Performance Metrics:**
|
||||
- NOMOUNT to MOUNT: ~30 seconds
|
||||
- Control file restore: ~10 seconds
|
||||
- Catalog backups: ~20 seconds
|
||||
- Database restore: ~8-10 minutes
|
||||
- Database recovery: ~2-3 minutes
|
||||
- OPEN RESETLOGS: ~1 minute
|
||||
- **Total Time: ~12-15 minutes**
|
||||
|
||||
### Automated Script Version
|
||||
|
||||
**Script:** `rman_restore_final.cmd`
|
||||
Location: `/mnt/e/proiecte/ROMFASTSQL/oracle/standby-server-scripts/rman_restore_final.cmd`
|
||||
|
||||
This CMD script automates all the above steps. Run on DR VM as Administrator:
|
||||
```cmd
|
||||
D:\oracle\scripts\rman_restore_final.cmd
|
||||
```
|
||||
|
||||
The script will:
|
||||
1. Shutdown database if running
|
||||
2. Startup NOMOUNT with correct PFILE
|
||||
3. Restore control file from correct backup piece (not autobackup)
|
||||
4. Mount database
|
||||
5. Catalog all backups in FRA
|
||||
6. Restore database with 2 parallel channels
|
||||
7. Recover database with NOREDO (no incremental)
|
||||
8. Open with RESETLOGS
|
||||
9. Create TEMP tablespace
|
||||
10. Verify database status
|
||||
|
||||
Log file: `D:\oracle\logs\rman_restore_final.log`
|
||||
|
||||
### 11. Document DR Restore Procedure 📝
|
||||
|
||||
After successful test, create:
|
||||
- **DR_RESTORE_PROCEDURE.md** - Step-by-step restore instructions
|
||||
- **DR_RUNBOOK.md** - Emergency runbook for DR event
|
||||
- Screenshots of successful restore
|
||||
- Performance metrics (restore time, verification steps)
|
||||
|
||||
### 12. Schedule Automated Testing 🗓️
|
||||
|
||||
- Monthly DR restore test (automated)
|
||||
- Quarterly full DR drill (manual verification)
|
||||
- Document test results in `D:\oracle\logs\dr_test_YYYYMMDD.log`
|
||||
|
||||
---
|
||||
|
||||
## 📋 PRIMARY SERVER CONFIGURATION (Reference)
|
||||
|
||||
**Server:** 10.0.20.36 (Windows Server)
|
||||
**Oracle Version:** 19c SE2 (19.3.0.0.0)
|
||||
**Database:** ROA, DBID: 1363569330, **non-CDB** (traditional architecture)
|
||||
|
||||
**Paths:**
|
||||
- ORACLE_HOME: `C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home`
|
||||
- ORACLE_BASE: `C:\Users\oracle`
|
||||
- Datafiles: `C:\Users\oracle\oradata\ROA\`
|
||||
- SYSTEM01.DBF
|
||||
- SYSAUX01.DBF
|
||||
- UNDOTBS01.DBF
|
||||
- TS_ROA.DBF (application tablespace)
|
||||
- USERS01.DBF
|
||||
- TEMP01.DBF (567 MB)
|
||||
- Control Files:
|
||||
- `C:\Users\oracle\oradata\ROA\control01.ctl`
|
||||
- `C:\Users\oracle\recovery_area\ROA\control02.ctl`
|
||||
- Redo Logs:
|
||||
- GROUP 1: `C:\Users\oracle\oradata\ROA\REDO01.LOG` (200 MB)
|
||||
- GROUP 2: `C:\Users\oracle\oradata\ROA\REDO02.LOG` (200 MB)
|
||||
- GROUP 3: `C:\Users\oracle\oradata\ROA\REDO03.LOG` (200 MB)
|
||||
- FRA: `C:\Users\Oracle\recovery_area\ROA`
|
||||
|
||||
**RMAN Configuration:**
|
||||
- Retention Policy: REDUNDANCY 2
|
||||
- Control File Autobackup: ON
|
||||
- Device Type: DISK, PARALLELISM 2, COMPRESSED BACKUPSET
|
||||
- Compression: BASIC
|
||||
|
||||
**Backup Schedule (Current - to be upgraded):**
|
||||
- FULL: Daily 02:30 AM (~6.32 GB compressed)
|
||||
- DIFFERENTIAL INCREMENTAL: Daily 14:00 (~50-120 MB) ⚠️ Not used in restore (causes UNDO corruption)
|
||||
- Retention: 2 days
|
||||
- Transfer to DR: Immediately after backup completes
|
||||
|
||||
**Planned Upgrade (see DR_UPGRADE_TO_CUMULATIVE_PLAN.md):**
|
||||
- FULL: Daily 02:30 AM (~6.32 GB compressed)
|
||||
- CUMULATIVE INCREMENTAL: Daily 13:00 + 18:00 (~150-400 MB each)
|
||||
- Retention: 2 days
|
||||
- Transfer to: Proxmox host (pveelite), mounted in VM when needed
|
||||
- **Target RPO:** 3-4 hours (vs current 24 hours)
|
||||
|
||||
**SSH:** OpenSSH Server on port 22122
|
||||
- SYSTEM user SSH key configured for automated transfers
|
||||
- Key: `ssh-rsa AAAAB3NzaC1yc...administrator@ROA-CARAPETRU2`
|
||||
|
||||
**Scheduled Tasks:**
|
||||
- Run as: `NT AUTHORITY\SYSTEM`
|
||||
- RMAN Full Backup + Transfer: Daily 02:30 AM
|
||||
- RMAN Incremental Backup + Transfer: Daily 14:00
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ KNOWN ISSUES & RESOLUTIONS
|
||||
|
||||
### 1. SSH Key Authentication - RESOLVED ✅
|
||||
**Issue:** Initial SSH key authentication failed with "Access Denied"
|
||||
**Root Cause:** File permissions on `administrators_authorized_keys` too restrictive
|
||||
**Resolution:**
|
||||
- Created script `fix_ssh_via_service.ps1`
|
||||
- Stops SSH service before modifying file
|
||||
- Uses `takeown` and `icacls` to set permissions
|
||||
- Both keys now working (user + SYSTEM)
|
||||
|
||||
### 2. Backup Transfer Directory Creation - RESOLVED ✅
|
||||
**Issue:** SCP transfers failed with exit code 1
|
||||
**Root Cause:** Directory `D:\oracle\backups\primary` didn't exist
|
||||
**Resolution:** Created directory manually via SSH
|
||||
**Note:** Transfer script command for creating directory had escaping issues
|
||||
|
||||
### 3. Oracle Silent Installation - RESOLVED ✅
|
||||
**Issue:** Silent installation failed with "username field is empty" (exit code 254)
|
||||
**Root Cause:** Windows silent install more complex than Linux
|
||||
**Resolution:** Used interactive GUI installation instead
|
||||
**Result:** Oracle 19c successfully installed, working perfectly
|
||||
|
||||
### 4. QEMU Guest Agent Intermittent Timeouts
|
||||
**Status:** Minor annoyance (NOT blocking)
|
||||
**Impact:** Cannot use `qm guest exec` reliably
|
||||
**Workaround:** Direct SSH access or Proxmox console
|
||||
**Fix:** Service QEMU-GA set to Automatic startup
|
||||
|
||||
---
|
||||
|
||||
## 📊 DR ARCHITECTURE SUMMARY
|
||||
|
||||
```
|
||||
PRIMARY (10.0.20.36) - Windows Server DR (10.0.20.37) - Windows 11 VM
|
||||
├─ Oracle 19c SE2 (19.3.0.0.0) ├─ Oracle 19c SE2 (19.3.0.0.0)
|
||||
├─ Database: ROA (LIVE, non-CDB) ├─ Database: ROA (OFFLINE, ready for restore)
|
||||
├─ RMAN Backups (FULL + INCR) ├─ Backup repository (6.32 GB)
|
||||
│ └─ Compressed BACKUPSET ├─ RMAN restore scripts
|
||||
│ └─ Listener configured and running
|
||||
└─ Transfer via SSH/SCP (automated)
|
||||
↓ port 22122, SYSTEM user key
|
||||
↓ Daily at 02:30 (FULL) and 14:00 (INCR)
|
||||
└─────────────────────────────────────────→ D:\oracle\backups\primary\
|
||||
Automated daily transfer
|
||||
950 Mbps network (~5 min for 6 GB)
|
||||
```
|
||||
|
||||
**RTO (Recovery Time Objective):** ~15 minutes
|
||||
- 2 min: Power on VM and wait for boot
|
||||
- 12 min: RMAN restore (database + recovery)
|
||||
- 1 min: Database open RESETLOGS and verify
|
||||
|
||||
**RPO (Recovery Point Objective - Current):**
|
||||
- Current: Only FULL backup used = **24 hours** (incremental not applied due to UNDO corruption issue)
|
||||
|
||||
**RPO (Planned after upgrade to CUMULATIVE):**
|
||||
- Target: FULL + latest CUMULATIVE = **3-4 hours**
|
||||
- Best case: 1 hour (disaster at 13:05, use 13:00 cumulative)
|
||||
- Worst case: 10.5 hours (disaster at 13:00, use 02:30 full only)
|
||||
|
||||
**Storage Requirements:**
|
||||
- VM disk: 500 GB total
|
||||
- Oracle installation: ~10 GB
|
||||
- Database (restored): ~15 GB
|
||||
- Backup repository: ~14 GB (2 days retention)
|
||||
- Free space: ~460 GB
|
||||
- Daily backup transfer: 6-7 GB (FULL) + 50-120 MB (INCR)
|
||||
|
||||
**Daily Resource Usage:**
|
||||
- VM powered OFF when not needed: **0 GB RAM, 0 CPU**
|
||||
- VM powered ON during DR event: **6 GB RAM, 4 CPU cores**
|
||||
- Network transfer: ~5-10 minutes/day at 950 Mbps
|
||||
|
||||
**Backup Retention:**
|
||||
- PRIMARY: 2 days in FRA
|
||||
- DR: 2 days in `D:\oracle\backups\primary`
|
||||
- Cleanup: Automated via transfer scripts
|
||||
|
||||
---
|
||||
|
||||
## 🎯 NEXT STEPS
|
||||
|
||||
### ✅ COMPLETED (Current Session):
|
||||
1. ✅ **RMAN Restore Tested** - Database successfully restored and operational
|
||||
2. ✅ **Database Verified** - All tablespaces, tables, data verified
|
||||
3. ✅ **Documented Results** - Restore time ~12-15 minutes
|
||||
4. ✅ **VM Shutdown** - Conserving resources
|
||||
|
||||
### 🔄 NEXT SESSION - Upgrade to CUMULATIVE Strategy:
|
||||
**Priority:** HIGH - Improves RPO from 24h to 3-4h
|
||||
|
||||
**See detailed plan:** `DR_UPGRADE_TO_CUMULATIVE_PLAN.md`
|
||||
|
||||
**Summary of changes:**
|
||||
1. 📦 **Configure Proxmox host storage** - Store backups on pveelite, mount in VM 109
|
||||
2. 🔄 **Convert DIFFERENTIAL → CUMULATIVE** - Add keyword to RMAN script
|
||||
3. ⏰ **Add second incremental** - Run at 13:00 + 18:00 (vs current 14:00 only)
|
||||
4. 📝 **Update transfer scripts** - Send to Proxmox host instead of VM
|
||||
5. 🗓️ **Update scheduled tasks** - Create 13:00 and 18:00 tasks
|
||||
6. 🧪 **Update restore script** - Read from mount point (E:\), handle cumulative backups
|
||||
7. ✅ **Test end-to-end** - Verify FULL + CUMULATIVE restore works
|
||||
|
||||
**Estimated time:** 2-3 hours
|
||||
**Recommended:** Saturday morning (low activity)
|
||||
|
||||
### Short Term (After Upgrade):
|
||||
1. 📄 **Update DR Runbook** - Include cumulative backup procedures
|
||||
2. 🧪 **Schedule Weekly Tests** - Automated Saturday morning DR tests
|
||||
3. 📊 **Create Monitoring** - Alert if backups fail to transfer
|
||||
4. 🔐 **Backup VM State** - Snapshot of configured DR VM
|
||||
|
||||
### Long Term:
|
||||
1. 🔄 **Automate Weekly Tests** - Script to test restore automatically
|
||||
2. 📈 **Performance Tuning** - Optimize restore speed if needed
|
||||
3. 🌐 **Network Failover** - DNS/routing changes for DR activation
|
||||
4. 📋 **Compliance** - Document DR procedures for audit
|
||||
|
||||
---
|
||||
|
||||
## 📞 SUPPORT CONTACTS & REFERENCES
|
||||
|
||||
**Documentation:**
|
||||
- Implementation plan: `oracle/standby-server-scripts/DR_WINDOWS_VM_IMPLEMENTATION_PLAN.md`
|
||||
- This status: `oracle/standby-server-scripts/DR_WINDOWS_VM_STATUS_2025-10-09.md`
|
||||
- Project directory: `/mnt/e/proiecte/ROMFASTSQL/oracle/standby-server-scripts/`
|
||||
|
||||
**Proxmox:**
|
||||
- Cluster: romfast
|
||||
- Nodes: pve1 (10.0.20.200), pvemini (10.0.20.201), pveelite (10.0.20.202)
|
||||
- VM 109 Commands:
|
||||
```bash
|
||||
qm status 109 # Check VM status
|
||||
qm start 109 # Power on VM
|
||||
qm stop 109 # Graceful shutdown
|
||||
qm shutdown 109 # Force shutdown
|
||||
qm console 109 # Open console (if needed)
|
||||
```
|
||||
|
||||
**Access Methods:**
|
||||
- **SSH (Preferred):** `ssh -p 22122 romfast@10.0.20.37`
|
||||
- Key authentication: ✅ Working
|
||||
- Password: Romfast2025! (if key fails)
|
||||
- **Proxmox Console:** Web UI → pveelite → VM 109 → Console
|
||||
- **RDP:** Not configured (SSH preferred for security)
|
||||
|
||||
**Oracle Quick Reference:**
|
||||
```powershell
|
||||
# On DR VM - Set environment
|
||||
$env:ORACLE_HOME = "C:\Users\Administrator\Downloads\WINDOWS.X64_193000_db_home"
|
||||
$env:ORACLE_SID = "ROA"
|
||||
$env:PATH = "$env:ORACLE_HOME\bin;$env:PATH"
|
||||
|
||||
# Connect to database
|
||||
sqlplus / as sysdba
|
||||
|
||||
# Check listener
|
||||
lsnrctl status
|
||||
|
||||
# Test TNS
|
||||
tnsping ROA
|
||||
```
|
||||
|
||||
**RMAN Quick Reference:**
|
||||
```bash
|
||||
# Connect to RMAN
|
||||
rman target /
|
||||
|
||||
# List backups
|
||||
LIST BACKUP SUMMARY;
|
||||
|
||||
# Validate backups
|
||||
VALIDATE BACKUPSET;
|
||||
|
||||
# Check database
|
||||
SELECT NAME, OPEN_MODE, LOG_MODE FROM V$DATABASE;
|
||||
```
|
||||
|
||||
**Useful Scripts Location:**
|
||||
- DR VM: `D:\oracle\scripts\`
|
||||
- PRIMARY: `D:\rman_backup\`
|
||||
- Project: `/mnt/e/proiecte/ROMFASTSQL/oracle/standby-server-scripts/`
|
||||
|
||||
**Oracle Documentation:**
|
||||
- RMAN Backup/Recovery: https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/
|
||||
- Windows Installation: https://docs.oracle.com/en/database/oracle/oracle-database/19/ntqrf/
|
||||
- Database Administrator's Guide: https://docs.oracle.com/en/database/oracle/oracle-database/19/admin/
|
||||
|
||||
---
|
||||
|
||||
## 📈 PROGRESS TRACKING
|
||||
|
||||
**Overall Status:** ~90% Complete
|
||||
**Estimated time to completion:** 30-60 minutes (RMAN restore test)
|
||||
**Blockers:** None - ready for final testing
|
||||
|
||||
**Completed:** 9/10 major tasks
|
||||
**Remaining:** 1/10 (RMAN restore test)
|
||||
|
||||
**Session Summary (2025-10-09):**
|
||||
- ✅ Fixed SSH key authentication (2 keys configured)
|
||||
- ✅ Installed Oracle 19c (interactive installation)
|
||||
- ✅ Configured Oracle Listener (running on port 1521)
|
||||
- ✅ Updated backup transfer scripts for Windows target
|
||||
- ✅ Added PRIMARY SYSTEM SSH key to DR VM
|
||||
- ✅ Successfully transferred 6.32 GB backup files
|
||||
- ✅ **COMPLETED RMAN restore testing - DATABASE FULLY OPERATIONAL**
|
||||
|
||||
**Time Invested:** ~5 hours total
|
||||
- Setup and configuration: ~1.5 hours
|
||||
- RMAN restore attempts and troubleshooting: ~3 hours
|
||||
- Successful restore and verification: ~30 minutes
|
||||
|
||||
**Critical Lessons Learned:**
|
||||
1. **Control file source matters** - Must use control file from same backup piece as datafiles, not autobackup
|
||||
2. **Incremental backups problematic** - Can cause UNDO corruption when restored on different platform state
|
||||
3. **FRA location critical** - Backups must be in Fast Recovery Area for RMAN auto-discovery
|
||||
4. **Memory constraints** - Windows reserves significant RAM, reduce Oracle memory_target accordingly
|
||||
5. **SET UNTIL TIME** - More reliable than SET UNTIL SCN for point-in-time recovery
|
||||
|
||||
**Final Database Metrics:**
|
||||
- Database: ROA (DBID: 1363569330)
|
||||
- Status: READ WRITE, OPEN
|
||||
- Tablespaces: 6 (all ONLINE)
|
||||
- Datafiles: 5
|
||||
- Application Owners: 69
|
||||
- Application Tables: 45,000+
|
||||
- Restore Time: 12-15 minutes (end-to-end)
|
||||
- Data Restored: 6.32 GB compressed → ~15 GB uncompressed
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-10-09 17:45 (Session completed)
|
||||
**Updated By:** Claude Code (Sonnet 4.5)
|
||||
**Status:** ✅ **RMAN RESTORE SUCCESSFUL - DR SYSTEM VALIDATED AND OPERATIONAL**
|
||||
|
||||
**Next Actions:**
|
||||
1. Shutdown database: `SHUTDOWN IMMEDIATE;`
|
||||
2. Power off VM to conserve resources: `qm stop 109`
|
||||
3. Implement CUMULATIVE backup strategy (see `DR_UPGRADE_TO_CUMULATIVE_PLAN.md`)
|
||||
4. Schedule weekly DR restore tests
|
||||
5. Create DR runbook for emergency procedures
|
||||
6. Monitor daily backup transfers from PRIMARY
|
||||
|
||||
**Important Notes:**
|
||||
- ⚠️ VM 109 partitions: C:, D:, E: (already used)
|
||||
- 📁 Mount point from host will appear as **F:\** (not E:\)
|
||||
- 🔄 For VM migration between nodes, see: `DR_VM_MIGRATION_GUIDE.md`
|
||||
Reference in New Issue
Block a user