thrillwiki_django_no_react/shared/scripts/unraid/README-template-deployment.md

# ThrillWiki Template-Based VM Deployment

This guide explains how to use the new **template-based VM deployment** system that dramatically speeds up VM creation by using a pre-configured Ubuntu template instead of autoinstall ISOs.

## Overview

### Traditional Approach (Slow)
- Create autoinstall ISO from scratch
- Boot VM from ISO (20-30 minutes)
- Wait for Ubuntu installation
- Configure system packages and dependencies

### Template Approach (Fast ⚡)
- Copy pre-configured VM disk from template
- Boot VM from template disk (2-5 minutes)
- System is already configured with Ubuntu, packages, and dependencies

## Prerequisites

1. **Template VM**: You must have a VM named `thrillwiki-template-ubuntu` on your Unraid server
2. **Template Configuration**: The template should be pre-configured with:
   - Ubuntu 24.04 LTS
   - Python 3, Git, PostgreSQL, Nginx
   - UV package manager (optional but recommended)
   - Basic system configuration

## Template VM Setup

### Creating the Template VM

1. **Create the template VM manually** on your Unraid server:
   - Name: `thrillwiki-template-ubuntu`
   - Install Ubuntu 24.04 LTS
   - Configure with 4GB RAM, 2 vCPUs (can be adjusted later)

2. **Configure the template** by SSH'ing into it and running:
   ```bash
   # Update system
   sudo apt update && sudo apt upgrade -y
   
   # Install required packages
   sudo apt install -y git curl build-essential python3-pip python3-venv
   sudo apt install -y postgresql postgresql-contrib nginx
   
   # Install UV (Python package manager)
   curl -LsSf https://astral.sh/uv/install.sh | sh
   source ~/.cargo/env
   
   # Create thrillwiki user with password 'thrillwiki'
   sudo useradd -m -s /bin/bash thrillwiki || true
   echo 'thrillwiki:thrillwiki' | sudo chpasswd
   sudo usermod -aG sudo thrillwiki
   
   # Setup SSH key for thrillwiki user 
   # First, generate your SSH key on your Mac:
   # ssh-keygen -t rsa -b 4096 -f ~/.ssh/thrillwiki_vm -N "" -C "thrillwiki-template-vm-access"
   # Then copy the public key to the template VM:
   sudo mkdir -p /home/thrillwiki/.ssh
   echo "YOUR_PUBLIC_KEY_FROM_~/.ssh/thrillwiki_vm.pub" | sudo tee /home/thrillwiki/.ssh/***REMOVED***
   sudo chown -R thrillwiki:thrillwiki /home/thrillwiki/.ssh
   sudo chmod 700 /home/thrillwiki/.ssh
   sudo chmod 600 /home/thrillwiki/.ssh/***REMOVED***
   
   # Configure PostgreSQL
   sudo systemctl enable postgresql
   sudo systemctl start postgresql
   
   # Configure Nginx
   sudo systemctl enable nginx
   
   # Clean up for template
   sudo apt autoremove -y
   sudo apt autoclean
   history -c && history -w
   
   # Shutdown template
   sudo shutdown now
   ```

3. **Verify template** is stopped and ready:
   ```bash
   ./template-utils.sh status  # Should show "shut off"
   ```

## Quick Start

### Step 0: Set Up SSH Key (First Time Only)

**IMPORTANT**: Before using template deployment, set up your SSH key:

```bash
# Generate and configure SSH key
./scripts/unraid/setup-ssh-key.sh

# Follow the instructions to add the public key to your template VM
```

See `TEMPLATE_VM_SETUP.md` for complete template VM setup instructions.

### Using the Utility Script

The easiest way to work with template VMs is using the utility script:

```bash
# Check if template is ready
./template-utils.sh check

# Get template information
./template-utils.sh info

# Deploy a new VM from template
./template-utils.sh deploy my-thrillwiki-vm

# Copy template to new VM (without full deployment)
./template-utils.sh copy my-vm-name

# List all template-based VMs
./template-utils.sh list
```

### Using Python Scripts Directly

For more control, use the Python scripts:

```bash
# Set environment variables
export UNRAID_HOST="your.unraid.server.ip"
export UNRAID_USER="root"
export VM_NAME="my-thrillwiki-vm"
export REPO_URL="owner/repository-name"

# Deploy VM from template
python3 main_template.py deploy

# Just create VM without ThrillWiki setup
python3 main_template.py setup

# Get VM status and IP
python3 main_template.py status
python3 main_template.py ip

# Manage template
python3 main_template.py template info
python3 main_template.py template check
```

## File Structure

### New Template-Based Files

```
scripts/unraid/
├── template_manager.py              # Template VM management
├── vm_manager_template.py          # Template-based VM manager  
├── main_template.py                # Template deployment orchestrator
├── template-utils.sh               # Quick utility commands
├── deploy-thrillwiki-template.sh   # Optimized deployment script
├── thrillwiki-vm-template-simple.xml  # VM XML without autoinstall ISO
└── README-template-deployment.md   # This documentation
```

### Original Files (Still Available)

```
scripts/unraid/
├── main.py                     # Original autoinstall approach
├── vm_manager.py              # Original VM manager
├── deploy-thrillwiki.sh       # Original deployment script
└── thrillwiki-vm-template.xml # Original XML with autoinstall
```

## Commands Reference

### Template Management

```bash
# Check template status
./template-utils.sh status
python3 template_manager.py check

# Get template information  
./template-utils.sh info
python3 template_manager.py info

# List VMs created from template
./template-utils.sh list
python3 template_manager.py list

# Update template instructions
./template-utils.sh update
python3 template_manager.py update
```

### VM Deployment

```bash
# Complete deployment (VM + ThrillWiki)
./template-utils.sh deploy VM_NAME
python3 main_template.py deploy

# VM setup only
python3 main_template.py setup

# Individual operations
python3 main_template.py create
python3 main_template.py start
python3 main_template.py stop
python3 main_template.py delete
```

### VM Information

```bash
# Get VM status
python3 main_template.py status

# Get VM IP and connection info
python3 main_template.py ip

# Get detailed VM information
python3 main_template.py info
```

## Environment Variables

Configure these in your `***REMOVED***.unraid` file or export them:

```bash
# Required
UNRAID_HOST="192.168.1.100"           # Your Unraid server IP
UNRAID_USER="root"                    # Unraid SSH user
VM_NAME="thrillwiki-vm"               # Name for new VM

# Optional VM Configuration
VM_MEMORY="4096"                      # Memory in MB
VM_VCPUS="2"                          # Number of vCPUs
VM_DISK_SIZE="50"                     # Disk size in GB (for reference)
VM_IP="dhcp"                          # IP configuration (dhcp or static IP)

# ThrillWiki Configuration
REPO_URL="owner/repository-name"      # GitHub repository
GITHUB_TOKEN="ghp_xxxxx"              # GitHub token (optional)
```

## Advantages of Template Approach

### Speed ⚡
- **VM Creation**: 2-5 minutes vs 20-30 minutes
- **Boot Time**: Instant boot vs full Ubuntu installation
- **Total Deployment**: ~10 minutes vs ~45 minutes

### Reliability 🔒
- **Pre-tested**: Template is already configured and tested
- **Consistent**: All VMs start from identical base
- **No Installation Failures**: No autoinstall ISO issues

### Efficiency 💾
- **Disk Space**: Copy-on-write QCOW2 format
- **Network**: No ISO downloads during deployment
- **Resources**: Less CPU usage during creation

## Troubleshooting

### Template Not Found
```
❌ Template VM disk not found at: /mnt/user/domains/thrillwiki-template-ubuntu/vdisk1.qcow2
```

**Solution**: Create the template VM first or verify the path.

### Template VM Running
```
⚠️ Template VM is currently running!
```

**Solution**: Stop the template VM before creating new instances:
```bash
ssh root@unraid-host "virsh shutdown thrillwiki-template-ubuntu"
```

### SSH Connection Issues
```
❌ Cannot connect to Unraid server
```

**Solutions**:
1. Verify `UNRAID_HOST` is correct
2. Ensure SSH key authentication is set up
3. Check network connectivity

### Template Disk Corruption

If template VM gets corrupted:
1. Start template VM and fix issues
2. Or recreate template VM from scratch
3. Update template: `./template-utils.sh update`

## Template Maintenance

### Updating the Template

Periodically update your template:

1. **Start template VM** on Unraid
2. **SSH into template** and update:
   ```bash
   sudo apt update && sudo apt upgrade -y
   sudo apt autoremove -y && sudo apt autoclean
   
   # Update UV if installed
   ~/.cargo/bin/uv --version
   
   # Clear history
   history -c && history -w
   ```
3. **Shutdown template VM**
4. **Verify update**: `./template-utils.sh check`

### Template Best Practices

- Keep template VM stopped when not maintaining it
- Update template monthly or before major deployments
- Test template by creating a test VM before important deployments
- Document any custom configurations in the template

## Migration Guide

### From Autoinstall to Template

1. **Create your template VM** following the setup guide above
2. **Test template deployment**:
   ```bash
   ./template-utils.sh deploy test-vm
   ```
3. **Update your automation scripts** to use template approach
4. **Keep autoinstall scripts** as backup for special cases

### Switching Between Approaches

You can use both approaches as needed:

```bash
# Template-based (fast)
python3 main_template.py deploy

# Autoinstall-based (traditional)  
python3 main.py setup
```

## Integration with CI/CD

The template approach integrates perfectly with your existing CI/CD:

```bash
# In your automation scripts
export UNRAID_HOST="your-server"
export VM_NAME="thrillwiki-$(date +%s)"
export REPO_URL="your-org/thrillwiki"

# Deploy quickly
./scripts/unraid/template-utils.sh deploy "$VM_NAME"

# VM is ready in minutes instead of 30+ minutes
```

## FAQ

**Q: Can I use both template and autoinstall approaches?**
A: Yes! Keep both. Use template for speed, autoinstall for special configurations.

**Q: How much disk space does template copying use?**
A: QCOW2 copy-on-write format means copies only store differences, saving space.

**Q: What if I need different Ubuntu versions?**
A: Create multiple template VMs (e.g., `thrillwiki-template-ubuntu-22`, `thrillwiki-template-ubuntu-24`).

**Q: Can I customize the template VM configuration?**
A: Yes! The template VM is just a regular VM. Customize it as needed.

**Q: Is this approach secure?**
A: Yes. Each VM gets a fresh copy and can be configured independently.

---

This template-based approach should make your VM deployments much faster and more reliable! 🚀