thrillwiki_django_no_react/shared/scripts/unraid/README-template-deployment.md

ThrillWiki Template-Based VM Deployment

This guide explains how to use the new template-based VM deployment system that dramatically speeds up VM creation by using a pre-configured Ubuntu template instead of autoinstall ISOs.

Overview

Traditional Approach (Slow)

• Create autoinstall ISO from scratch
• Boot VM from ISO (20-30 minutes)
• Wait for Ubuntu installation
• Configure system packages and dependencies

Template Approach (Fast ⚡)

• Copy pre-configured VM disk from template
• Boot VM from template disk (2-5 minutes)
• System is already configured with Ubuntu, packages, and dependencies

Prerequisites

1. Template VM: You must have a VM named thrillwiki-template-ubuntu on your Unraid server
2. Template Configuration: The template should be pre-configured with:
   • Ubuntu 24.04 LTS
   • Python 3, Git, PostgreSQL, Nginx
   • UV package manager (optional but recommended)
   • Basic system configuration

Template VM Setup

Creating the Template VM

1. Create the template VM manually on your Unraid server:

   • Name: thrillwiki-template-ubuntu
   • Install Ubuntu 24.04 LTS
   • Configure with 4GB RAM, 2 vCPUs (can be adjusted later)

2. Configure the template by SSH'ing into it and running:

   # Update system
   sudo apt update && sudo apt upgrade -y
   
   # Install required packages
   sudo apt install -y git curl build-essential python3-pip python3-venv
   sudo apt install -y postgresql postgresql-contrib nginx
   
   # Install UV (Python package manager)
   curl -LsSf https://astral.sh/uv/install.sh | sh
   source ~/.cargo/env
   
   # Create thrillwiki user with password 'thrillwiki'
   sudo useradd -m -s /bin/bash thrillwiki || true
   echo 'thrillwiki:thrillwiki' | sudo chpasswd
   sudo usermod -aG sudo thrillwiki
   
   # Setup SSH key for thrillwiki user 
   # First, generate your SSH key on your Mac:
   # ssh-keygen -t rsa -b 4096 -f ~/.ssh/thrillwiki_vm -N "" -C "thrillwiki-template-vm-access"
   # Then copy the public key to the template VM:
   sudo mkdir -p /home/thrillwiki/.ssh
   echo "YOUR_PUBLIC_KEY_FROM_~/.ssh/thrillwiki_vm.pub" | sudo tee /home/thrillwiki/.ssh/***REMOVED***
   sudo chown -R thrillwiki:thrillwiki /home/thrillwiki/.ssh
   sudo chmod 700 /home/thrillwiki/.ssh
   sudo chmod 600 /home/thrillwiki/.ssh/***REMOVED***
   
   # Configure PostgreSQL
   sudo systemctl enable postgresql
   sudo systemctl start postgresql
   
   # Configure Nginx
   sudo systemctl enable nginx
   
   # Clean up for template
   sudo apt autoremove -y
   sudo apt autoclean
   history -c && history -w
   
   # Shutdown template
   sudo shutdown now
   

3. Verify template is stopped and ready:

   ./template-utils.sh status  # Should show "shut off"
   

Quick Start

Step 0: Set Up SSH Key (First Time Only)

IMPORTANT: Before using template deployment, set up your SSH key:

# Generate and configure SSH key
./scripts/unraid/setup-ssh-key.sh

# Follow the instructions to add the public key to your template VM

See TEMPLATE_VM_SETUP.md for complete template VM setup instructions.

Using the Utility Script

The easiest way to work with template VMs is using the utility script:

# Check if template is ready
./template-utils.sh check

# Get template information
./template-utils.sh info

# Deploy a new VM from template
./template-utils.sh deploy my-thrillwiki-vm

# Copy template to new VM (without full deployment)
./template-utils.sh copy my-vm-name

# List all template-based VMs
./template-utils.sh list

Using Python Scripts Directly

For more control, use the Python scripts:

# Set environment variables
export UNRAID_HOST="your.unraid.server.ip"
export UNRAID_USER="root"
export VM_NAME="my-thrillwiki-vm"
export REPO_URL="owner/repository-name"

# Deploy VM from template
python3 main_template.py deploy

# Just create VM without ThrillWiki setup
python3 main_template.py setup

# Get VM status and IP
python3 main_template.py status
python3 main_template.py ip

# Manage template
python3 main_template.py template info
python3 main_template.py template check

File Structure

New Template-Based Files

scripts/unraid/
├── template_manager.py              # Template VM management
├── vm_manager_template.py          # Template-based VM manager  
├── main_template.py                # Template deployment orchestrator
├── template-utils.sh               # Quick utility commands
├── deploy-thrillwiki-template.sh   # Optimized deployment script
├── thrillwiki-vm-template-simple.xml  # VM XML without autoinstall ISO
└── README-template-deployment.md   # This documentation

Original Files (Still Available)

scripts/unraid/
├── main.py                     # Original autoinstall approach
├── vm_manager.py              # Original VM manager
├── deploy-thrillwiki.sh       # Original deployment script
└── thrillwiki-vm-template.xml # Original XML with autoinstall

Commands Reference

Template Management

# Check template status
./template-utils.sh status
python3 template_manager.py check

# Get template information  
./template-utils.sh info
python3 template_manager.py info

# List VMs created from template
./template-utils.sh list
python3 template_manager.py list

# Update template instructions
./template-utils.sh update
python3 template_manager.py update

VM Deployment

# Complete deployment (VM + ThrillWiki)
./template-utils.sh deploy VM_NAME
python3 main_template.py deploy

# VM setup only
python3 main_template.py setup

# Individual operations
python3 main_template.py create
python3 main_template.py start
python3 main_template.py stop
python3 main_template.py delete

VM Information

# Get VM status
python3 main_template.py status

# Get VM IP and connection info
python3 main_template.py ip

# Get detailed VM information
python3 main_template.py info

Environment Variables

Configure these in your ***REMOVED***.unraid file or export them:

# Required
UNRAID_HOST="192.168.1.100"           # Your Unraid server IP
UNRAID_USER="root"                    # Unraid SSH user
VM_NAME="thrillwiki-vm"               # Name for new VM

# Optional VM Configuration
VM_MEMORY="4096"                      # Memory in MB
VM_VCPUS="2"                          # Number of vCPUs
VM_DISK_SIZE="50"                     # Disk size in GB (for reference)
VM_IP="dhcp"                          # IP configuration (dhcp or static IP)

# ThrillWiki Configuration
REPO_URL="owner/repository-name"      # GitHub repository
GITHUB_TOKEN="ghp_xxxxx"              # GitHub token (optional)

Advantages of Template Approach

Speed ⚡

• VM Creation: 2-5 minutes vs 20-30 minutes
• Boot Time: Instant boot vs full Ubuntu installation
• Total Deployment: ~10 minutes vs ~45 minutes

Reliability 🔒

• Pre-tested: Template is already configured and tested
• Consistent: All VMs start from identical base
• No Installation Failures: No autoinstall ISO issues

Efficiency 💾

• Disk Space: Copy-on-write QCOW2 format
• Network: No ISO downloads during deployment
• Resources: Less CPU usage during creation

Troubleshooting

Template Not Found

❌ Template VM disk not found at: /mnt/user/domains/thrillwiki-template-ubuntu/vdisk1.qcow2

Solution: Create the template VM first or verify the path.

Template VM Running

⚠️ Template VM is currently running!

Solution: Stop the template VM before creating new instances:

ssh root@unraid-host "virsh shutdown thrillwiki-template-ubuntu"

SSH Connection Issues

❌ Cannot connect to Unraid server

Solutions:

1. Verify UNRAID_HOST is correct
2. Ensure SSH key authentication is set up
3. Check network connectivity

Template Disk Corruption

If template VM gets corrupted:

1. Start template VM and fix issues
2. Or recreate template VM from scratch
3. Update template: ./template-utils.sh update

Template Maintenance

Updating the Template

Periodically update your template:

1. Start template VM on Unraid
2. SSH into template and update:

   sudo apt update && sudo apt upgrade -y
   sudo apt autoremove -y && sudo apt autoclean
   
   # Update UV if installed
   ~/.cargo/bin/uv --version
   
   # Clear history
   history -c && history -w
   

3. Shutdown template VM
4. Verify update: ./template-utils.sh check

Template Best Practices

• Keep template VM stopped when not maintaining it
• Update template monthly or before major deployments
• Test template by creating a test VM before important deployments
• Document any custom configurations in the template

Migration Guide

From Autoinstall to Template

1. Create your template VM following the setup guide above
2. Test template deployment:

   ./template-utils.sh deploy test-vm
   

3. Update your automation scripts to use template approach
4. Keep autoinstall scripts as backup for special cases

Switching Between Approaches

You can use both approaches as needed:

# Template-based (fast)
python3 main_template.py deploy

# Autoinstall-based (traditional)  
python3 main.py setup

Integration with CI/CD

The template approach integrates perfectly with your existing CI/CD:

# In your automation scripts
export UNRAID_HOST="your-server"
export VM_NAME="thrillwiki-$(date +%s)"
export REPO_URL="your-org/thrillwiki"

# Deploy quickly
./scripts/unraid/template-utils.sh deploy "$VM_NAME"

# VM is ready in minutes instead of 30+ minutes

FAQ

Q: Can I use both template and autoinstall approaches? A: Yes! Keep both. Use template for speed, autoinstall for special configurations.

Q: How much disk space does template copying use? A: QCOW2 copy-on-write format means copies only store differences, saving space.

Q: What if I need different Ubuntu versions? A: Create multiple template VMs (e.g., thrillwiki-template-ubuntu-22, thrillwiki-template-ubuntu-24).

Q: Can I customize the template VM configuration? A: Yes! The template VM is just a regular VM. Customize it as needed.

Q: Is this approach secure? A: Yes. Each VM gets a fresh copy and can be configured independently.

---

This template-based approach should make your VM deployments much faster and more reliable! 🚀