mirror of
https://github.com/pacnpal/thrillwiki_django_no_react.git
synced 2025-12-20 12:51:09 -05:00
feat: complete monorepo structure with frontend and shared resources
- Add complete backend/ directory with full Django application - Add frontend/ directory with Vite + TypeScript setup ready for Next.js - Add comprehensive shared/ directory with: - Complete documentation and memory-bank archives - Media files and avatars (letters, park/ride images) - Deployment scripts and automation tools - Shared types and utilities - Add architecture/ directory with migration guides - Configure pnpm workspace for monorepo development - Update .gitignore to exclude .django_tailwind_cli/ build artifacts - Preserve all historical documentation in shared/docs/memory-bank/ - Set up proper structure for full-stack development with shared resources
This commit is contained in:
pacnpal
385
shared/scripts/unraid/README-template-deployment.md
Normal file
385
shared/scripts/unraid/README-template-deployment.md
Normal file
@@ -0,0 +1,385 @@
|
||||
# ThrillWiki Template-Based VM Deployment
|
||||
|
||||
This guide explains how to use the new **template-based VM deployment** system that dramatically speeds up VM creation by using a pre-configured Ubuntu template instead of autoinstall ISOs.
|
||||
|
||||
## Overview
|
||||
|
||||
### Traditional Approach (Slow)
|
||||
- Create autoinstall ISO from scratch
|
||||
- Boot VM from ISO (20-30 minutes)
|
||||
- Wait for Ubuntu installation
|
||||
- Configure system packages and dependencies
|
||||
|
||||
### Template Approach (Fast ⚡)
|
||||
- Copy pre-configured VM disk from template
|
||||
- Boot VM from template disk (2-5 minutes)
|
||||
- System is already configured with Ubuntu, packages, and dependencies
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Template VM**: You must have a VM named `thrillwiki-template-ubuntu` on your Unraid server
|
||||
2. **Template Configuration**: The template should be pre-configured with:
|
||||
- Ubuntu 24.04 LTS
|
||||
- Python 3, Git, PostgreSQL, Nginx
|
||||
- UV package manager (optional but recommended)
|
||||
- Basic system configuration
|
||||
|
||||
## Template VM Setup
|
||||
|
||||
### Creating the Template VM
|
||||
|
||||
1. **Create the template VM manually** on your Unraid server:
|
||||
- Name: `thrillwiki-template-ubuntu`
|
||||
- Install Ubuntu 24.04 LTS
|
||||
- Configure with 4GB RAM, 2 vCPUs (can be adjusted later)
|
||||
|
||||
2. **Configure the template** by SSH'ing into it and running:
|
||||
```bash
|
||||
# Update system
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Install required packages
|
||||
sudo apt install -y git curl build-essential python3-pip python3-venv
|
||||
sudo apt install -y postgresql postgresql-contrib nginx
|
||||
|
||||
# Install UV (Python package manager)
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
source ~/.cargo/env
|
||||
|
||||
# Create thrillwiki user with password 'thrillwiki'
|
||||
sudo useradd -m -s /bin/bash thrillwiki || true
|
||||
echo 'thrillwiki:thrillwiki' | sudo chpasswd
|
||||
sudo usermod -aG sudo thrillwiki
|
||||
|
||||
# Setup SSH key for thrillwiki user
|
||||
# First, generate your SSH key on your Mac:
|
||||
# ssh-keygen -t rsa -b 4096 -f ~/.ssh/thrillwiki_vm -N "" -C "thrillwiki-template-vm-access"
|
||||
# Then copy the public key to the template VM:
|
||||
sudo mkdir -p /home/thrillwiki/.ssh
|
||||
echo "YOUR_PUBLIC_KEY_FROM_~/.ssh/thrillwiki_vm.pub" | sudo tee /home/thrillwiki/.ssh/***REMOVED***
|
||||
sudo chown -R thrillwiki:thrillwiki /home/thrillwiki/.ssh
|
||||
sudo chmod 700 /home/thrillwiki/.ssh
|
||||
sudo chmod 600 /home/thrillwiki/.ssh/***REMOVED***
|
||||
|
||||
# Configure PostgreSQL
|
||||
sudo systemctl enable postgresql
|
||||
sudo systemctl start postgresql
|
||||
|
||||
# Configure Nginx
|
||||
sudo systemctl enable nginx
|
||||
|
||||
# Clean up for template
|
||||
sudo apt autoremove -y
|
||||
sudo apt autoclean
|
||||
history -c && history -w
|
||||
|
||||
# Shutdown template
|
||||
sudo shutdown now
|
||||
```
|
||||
|
||||
3. **Verify template** is stopped and ready:
|
||||
```bash
|
||||
./template-utils.sh status # Should show "shut off"
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 0: Set Up SSH Key (First Time Only)
|
||||
|
||||
**IMPORTANT**: Before using template deployment, set up your SSH key:
|
||||
|
||||
```bash
|
||||
# Generate and configure SSH key
|
||||
./scripts/unraid/setup-ssh-key.sh
|
||||
|
||||
# Follow the instructions to add the public key to your template VM
|
||||
```
|
||||
|
||||
See `TEMPLATE_VM_SETUP.md` for complete template VM setup instructions.
|
||||
|
||||
### Using the Utility Script
|
||||
|
||||
The easiest way to work with template VMs is using the utility script:
|
||||
|
||||
```bash
|
||||
# Check if template is ready
|
||||
./template-utils.sh check
|
||||
|
||||
# Get template information
|
||||
./template-utils.sh info
|
||||
|
||||
# Deploy a new VM from template
|
||||
./template-utils.sh deploy my-thrillwiki-vm
|
||||
|
||||
# Copy template to new VM (without full deployment)
|
||||
./template-utils.sh copy my-vm-name
|
||||
|
||||
# List all template-based VMs
|
||||
./template-utils.sh list
|
||||
```
|
||||
|
||||
### Using Python Scripts Directly
|
||||
|
||||
For more control, use the Python scripts:
|
||||
|
||||
```bash
|
||||
# Set environment variables
|
||||
export UNRAID_HOST="your.unraid.server.ip"
|
||||
export UNRAID_USER="root"
|
||||
export VM_NAME="my-thrillwiki-vm"
|
||||
export REPO_URL="owner/repository-name"
|
||||
|
||||
# Deploy VM from template
|
||||
python3 main_template.py deploy
|
||||
|
||||
# Just create VM without ThrillWiki setup
|
||||
python3 main_template.py setup
|
||||
|
||||
# Get VM status and IP
|
||||
python3 main_template.py status
|
||||
python3 main_template.py ip
|
||||
|
||||
# Manage template
|
||||
python3 main_template.py template info
|
||||
python3 main_template.py template check
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
### New Template-Based Files
|
||||
|
||||
```
|
||||
scripts/unraid/
|
||||
├── template_manager.py # Template VM management
|
||||
├── vm_manager_template.py # Template-based VM manager
|
||||
├── main_template.py # Template deployment orchestrator
|
||||
├── template-utils.sh # Quick utility commands
|
||||
├── deploy-thrillwiki-template.sh # Optimized deployment script
|
||||
├── thrillwiki-vm-template-simple.xml # VM XML without autoinstall ISO
|
||||
└── README-template-deployment.md # This documentation
|
||||
```
|
||||
|
||||
### Original Files (Still Available)
|
||||
|
||||
```
|
||||
scripts/unraid/
|
||||
├── main.py # Original autoinstall approach
|
||||
├── vm_manager.py # Original VM manager
|
||||
├── deploy-thrillwiki.sh # Original deployment script
|
||||
└── thrillwiki-vm-template.xml # Original XML with autoinstall
|
||||
```
|
||||
|
||||
## Commands Reference
|
||||
|
||||
### Template Management
|
||||
|
||||
```bash
|
||||
# Check template status
|
||||
./template-utils.sh status
|
||||
python3 template_manager.py check
|
||||
|
||||
# Get template information
|
||||
./template-utils.sh info
|
||||
python3 template_manager.py info
|
||||
|
||||
# List VMs created from template
|
||||
./template-utils.sh list
|
||||
python3 template_manager.py list
|
||||
|
||||
# Update template instructions
|
||||
./template-utils.sh update
|
||||
python3 template_manager.py update
|
||||
```
|
||||
|
||||
### VM Deployment
|
||||
|
||||
```bash
|
||||
# Complete deployment (VM + ThrillWiki)
|
||||
./template-utils.sh deploy VM_NAME
|
||||
python3 main_template.py deploy
|
||||
|
||||
# VM setup only
|
||||
python3 main_template.py setup
|
||||
|
||||
# Individual operations
|
||||
python3 main_template.py create
|
||||
python3 main_template.py start
|
||||
python3 main_template.py stop
|
||||
python3 main_template.py delete
|
||||
```
|
||||
|
||||
### VM Information
|
||||
|
||||
```bash
|
||||
# Get VM status
|
||||
python3 main_template.py status
|
||||
|
||||
# Get VM IP and connection info
|
||||
python3 main_template.py ip
|
||||
|
||||
# Get detailed VM information
|
||||
python3 main_template.py info
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Configure these in your `***REMOVED***.unraid` file or export them:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
UNRAID_HOST="192.168.1.100" # Your Unraid server IP
|
||||
UNRAID_USER="root" # Unraid SSH user
|
||||
VM_NAME="thrillwiki-vm" # Name for new VM
|
||||
|
||||
# Optional VM Configuration
|
||||
VM_MEMORY="4096" # Memory in MB
|
||||
VM_VCPUS="2" # Number of vCPUs
|
||||
VM_DISK_SIZE="50" # Disk size in GB (for reference)
|
||||
VM_IP="dhcp" # IP configuration (dhcp or static IP)
|
||||
|
||||
# ThrillWiki Configuration
|
||||
REPO_URL="owner/repository-name" # GitHub repository
|
||||
GITHUB_TOKEN="ghp_xxxxx" # GitHub token (optional)
|
||||
```
|
||||
|
||||
## Advantages of Template Approach
|
||||
|
||||
### Speed ⚡
|
||||
- **VM Creation**: 2-5 minutes vs 20-30 minutes
|
||||
- **Boot Time**: Instant boot vs full Ubuntu installation
|
||||
- **Total Deployment**: ~10 minutes vs ~45 minutes
|
||||
|
||||
### Reliability 🔒
|
||||
- **Pre-tested**: Template is already configured and tested
|
||||
- **Consistent**: All VMs start from identical base
|
||||
- **No Installation Failures**: No autoinstall ISO issues
|
||||
|
||||
### Efficiency 💾
|
||||
- **Disk Space**: Copy-on-write QCOW2 format
|
||||
- **Network**: No ISO downloads during deployment
|
||||
- **Resources**: Less CPU usage during creation
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Template Not Found
|
||||
```
|
||||
❌ Template VM disk not found at: /mnt/user/domains/thrillwiki-template-ubuntu/vdisk1.qcow2
|
||||
```
|
||||
|
||||
**Solution**: Create the template VM first or verify the path.
|
||||
|
||||
### Template VM Running
|
||||
```
|
||||
⚠️ Template VM is currently running!
|
||||
```
|
||||
|
||||
**Solution**: Stop the template VM before creating new instances:
|
||||
```bash
|
||||
ssh root@unraid-host "virsh shutdown thrillwiki-template-ubuntu"
|
||||
```
|
||||
|
||||
### SSH Connection Issues
|
||||
```
|
||||
❌ Cannot connect to Unraid server
|
||||
```
|
||||
|
||||
**Solutions**:
|
||||
1. Verify `UNRAID_HOST` is correct
|
||||
2. Ensure SSH key authentication is set up
|
||||
3. Check network connectivity
|
||||
|
||||
### Template Disk Corruption
|
||||
|
||||
If template VM gets corrupted:
|
||||
1. Start template VM and fix issues
|
||||
2. Or recreate template VM from scratch
|
||||
3. Update template: `./template-utils.sh update`
|
||||
|
||||
## Template Maintenance
|
||||
|
||||
### Updating the Template
|
||||
|
||||
Periodically update your template:
|
||||
|
||||
1. **Start template VM** on Unraid
|
||||
2. **SSH into template** and update:
|
||||
```bash
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
sudo apt autoremove -y && sudo apt autoclean
|
||||
|
||||
# Update UV if installed
|
||||
~/.cargo/bin/uv --version
|
||||
|
||||
# Clear history
|
||||
history -c && history -w
|
||||
```
|
||||
3. **Shutdown template VM**
|
||||
4. **Verify update**: `./template-utils.sh check`
|
||||
|
||||
### Template Best Practices
|
||||
|
||||
- Keep template VM stopped when not maintaining it
|
||||
- Update template monthly or before major deployments
|
||||
- Test template by creating a test VM before important deployments
|
||||
- Document any custom configurations in the template
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### From Autoinstall to Template
|
||||
|
||||
1. **Create your template VM** following the setup guide above
|
||||
2. **Test template deployment**:
|
||||
```bash
|
||||
./template-utils.sh deploy test-vm
|
||||
```
|
||||
3. **Update your automation scripts** to use template approach
|
||||
4. **Keep autoinstall scripts** as backup for special cases
|
||||
|
||||
### Switching Between Approaches
|
||||
|
||||
You can use both approaches as needed:
|
||||
|
||||
```bash
|
||||
# Template-based (fast)
|
||||
python3 main_template.py deploy
|
||||
|
||||
# Autoinstall-based (traditional)
|
||||
python3 main.py setup
|
||||
```
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
The template approach integrates perfectly with your existing CI/CD:
|
||||
|
||||
```bash
|
||||
# In your automation scripts
|
||||
export UNRAID_HOST="your-server"
|
||||
export VM_NAME="thrillwiki-$(date +%s)"
|
||||
export REPO_URL="your-org/thrillwiki"
|
||||
|
||||
# Deploy quickly
|
||||
./scripts/unraid/template-utils.sh deploy "$VM_NAME"
|
||||
|
||||
# VM is ready in minutes instead of 30+ minutes
|
||||
```
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I use both template and autoinstall approaches?**
|
||||
A: Yes! Keep both. Use template for speed, autoinstall for special configurations.
|
||||
|
||||
**Q: How much disk space does template copying use?**
|
||||
A: QCOW2 copy-on-write format means copies only store differences, saving space.
|
||||
|
||||
**Q: What if I need different Ubuntu versions?**
|
||||
A: Create multiple template VMs (e.g., `thrillwiki-template-ubuntu-22`, `thrillwiki-template-ubuntu-24`).
|
||||
|
||||
**Q: Can I customize the template VM configuration?**
|
||||
A: Yes! The template VM is just a regular VM. Customize it as needed.
|
||||
|
||||
**Q: Is this approach secure?**
|
||||
A: Yes. Each VM gets a fresh copy and can be configured independently.
|
||||
|
||||
---
|
||||
|
||||
This template-based approach should make your VM deployments much faster and more reliable! 🚀
|
||||
Reference in New Issue
Block a user