thrillwiki_django_no_react/CI_README.md

ThrillWiki CI/CD System

This repository includes a complete automated CI/CD system that creates a Linux VM on Unraid and automatically deploys ThrillWiki when commits are pushed to GitHub.

🚀 Complete Automation (Unraid)

For full automation including VM creation on Unraid:

./scripts/unraid/setup-complete-automation.sh

This single command will:

• ✅ Create and configure VM on Unraid
• ✅ Install Ubuntu Server with all dependencies
• ✅ Deploy ThrillWiki application
• ✅ Set up automated CI/CD pipeline
• ✅ Configure webhook listener
• ✅ Test the entire system

Manual Setup (Any Linux VM)

For manual setup on existing Linux VMs:

./scripts/setup-vm-ci.sh

System Components

📁 Files Created

scripts/
├── ci-start.sh                      # Local development server startup
├── webhook-listener.py              # GitHub webhook listener
├── vm-deploy.sh                    # VM deployment script
├── setup-vm-ci.sh                  # Manual VM setup script
├── unraid/
│   ├── vm-manager.py               # Unraid VM management
│   └── setup-complete-automation.sh # Complete automation
└── systemd/
    ├── thrillwiki.service           # Django app service
    └── thrillwiki-webhook.service   # Webhook listener service

docs/
├── VM_DEPLOYMENT_SETUP.md          # Manual setup documentation
└── UNRAID_COMPLETE_AUTOMATION.md   # Complete automation guide

🔄 Deployment Flow

Complete Automation:

GitHub Push → Webhook → Local Listener → SSH → Unraid VM → Deploy & Restart

Manual Setup:

GitHub Push → Webhook → Local Listener → SSH to VM → Deploy Script → Server Restart

Features

• Complete VM Automation: Automatically creates VMs on Unraid
• Automatic Deployment: Deploys on push to main branch
• Health Checks: Verifies deployment success
• Rollback Support: Automatic rollback on deployment failure
• Service Management: Systemd integration for reliable service management
• Database Setup: Automated PostgreSQL configuration
• Logging: Comprehensive logging for debugging
• Security: SSH key authentication and webhook secrets
• One-Command Setup: Full automation with single script

Usage

Complete Automation (Recommended)

For Unraid users, run the complete automation:

./scripts/unraid/setup-complete-automation.sh

After setup, start the webhook listener:

./start-webhook.sh

Local Development

Start the local development server:

./scripts/ci-start.sh

VM Management (Unraid)

# Check VM status
python3 scripts/unraid/vm-manager.py status

# Start/stop VM
python3 scripts/unraid/vm-manager.py start
python3 scripts/unraid/vm-manager.py stop

# Get VM IP
python3 scripts/unraid/vm-manager.py ip

Service Management

On the VM:

# Check status
ssh thrillwiki-vm "./scripts/vm-deploy.sh status"

# Restart service
ssh thrillwiki-vm "./scripts/vm-deploy.sh restart"

# View logs
ssh thrillwiki-vm "journalctl -u thrillwiki -f"

Manual VM Deployment

Deploy to VM manually:

ssh thrillwiki-vm "cd thrillwiki && ./scripts/vm-deploy.sh"

Configuration

Automated Configuration

The complete automation script creates all necessary configuration files:

• ***REMOVED***.unraid - Unraid VM configuration
• ***REMOVED***.webhook - Webhook listener configuration
• SSH keys and configuration
• Service configurations

Manual Environment Variables

For manual setup, create ***REMOVED***.webhook file:

WEBHOOK_PORT=9000
WEBHOOK_SECRET=your_secret_here
VM_HOST=your_vm_ip
VM_USER=ubuntu
VM_KEY_PATH=/path/to/ssh/key
VM_PROJECT_PATH=/home/ubuntu/thrillwiki
REPO_URL=https://github.com/username/repo.git
DEPLOY_BRANCH=main

GitHub Webhook

Configure in your GitHub repository:

• URL: http://YOUR_PUBLIC_IP:9000/webhook
• Content Type: application/json
• Secret: Your webhook secret
• Events: Push events

Requirements

For Complete Automation

• Local Machine: Python 3.8+, SSH client
• Unraid Server: 6.8+ with VM support
• Resources: 4GB RAM, 50GB disk minimum
• Ubuntu ISO: Ubuntu Server 22.04 in /mnt/user/isos/

For Manual Setup

• Local Machine: Python 3.8+, SSH access to VM, Public IP
• Linux VM: Ubuntu 20.04+, Python 3.8+, UV package manager, Git, SSH server

Troubleshooting

Complete Automation Issues

1. VM Creation Fails

   # Check Unraid VM support
   ssh unraid "virsh list --all"
   
   # Verify Ubuntu ISO exists
   ssh unraid "ls -la /mnt/user/isos/ubuntu-*.iso"
   

2. VM Won't Start

   # Check VM status
   python3 scripts/unraid/vm-manager.py status
   
   # Check Unraid logs
   ssh unraid "tail -f /var/log/libvirt/qemu/thrillwiki-vm.log"
   

General Issues

1. SSH Connection Failed

   # Check SSH key permissions
   chmod 600 ~/.ssh/thrillwiki_vm
   
   # Test connection
   ssh thrillwiki-vm
   

2. Webhook Not Receiving Events

   # Check if port is open
   sudo ufw allow 9000
   
   # Verify webhook URL in GitHub
   curl -X GET http://localhost:9000/health
   

3. Service Won't Start

   # Check service logs
   ssh thrillwiki-vm "journalctl -u thrillwiki --no-pager"
   
   # Manual start
   ssh thrillwiki-vm "cd thrillwiki && ./scripts/ci-start.sh"
   

Logs

• Setup logs: logs/unraid-automation.log
• Local webhook: logs/webhook.log
• VM deployment: logs/deploy.log (on VM)
• Django server: logs/django.log (on VM)
• System logs: journalctl -u thrillwiki -f (on VM)

Security Notes

• Automated SSH key generation and management
• Dedicated keys for each connection (VM access, Unraid access)
• No password authentication
• Systemd security features enabled
• Firewall configuration support
• Secret management in environment files

Documentation

• Complete Automation: docs/UNRAID_COMPLETE_AUTOMATION.md
• Manual Setup: docs/VM_DEPLOYMENT_SETUP.md

---

Quick Start Summary

For Unraid Users (Complete Automation)

# One command to set up everything
./scripts/unraid/setup-complete-automation.sh

# Start webhook listener
./start-webhook.sh

# Push commits to auto-deploy!

For Existing VM Users

# Manual setup
./scripts/setup-vm-ci.sh

# Configure webhook and push to deploy

The system will automatically deploy your Django application whenever you push commits to the main branch! 🚀