thrillwiki_django_no_react/docs/VM_DEPLOYMENT_SETUP.md

# ThrillWiki VM Deployment Setup Guide

This guide explains how to set up a local CI/CD system that automatically deploys ThrillWiki to a Linux VM when commits are pushed to GitHub.

## System Overview

The deployment system consists of three main components:

1. **Local CI Start Script** (`scripts/ci-start.sh`) - Starts the Django server locally
2. **GitHub Webhook Listener** (`scripts/webhook-listener.py`) - Listens for GitHub push events
3. **VM Deployment Script** (`scripts/vm-deploy.sh`) - Deploys code changes to the Linux VM

## Architecture Flow

```
GitHub Push → Webhook → Local Listener → SSH to VM → Deploy Script → Restart Server
```

## Prerequisites

### Local Machine (Webhook Listener Host)
- Python 3.8+
- SSH access to the Linux VM
- Git repository with webhook access

### Linux VM (Deployment Target)
- Ubuntu 20.04+ (recommended)
- Python 3.8+
- UV package manager
- Git
- PostgreSQL (if using database)
- SSH server running
- Sudo access for the deployment user

## Step 1: Linux VM Setup

### 1.1 Create Deployment User

```bash
# On the Linux VM
sudo adduser ubuntu
sudo usermod -aG sudo ubuntu
su - ubuntu
```

### 1.2 Install Required Software

```bash
# Update system
sudo apt update && sudo apt upgrade -y

# Install essential packages
sudo apt install -y git curl build-essential python3-pip python3-venv postgresql postgresql-contrib nginx

# Install UV package manager
curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.cargo/env
```

### 1.3 Set up SSH Keys

```bash
# Generate SSH key on local machine
ssh-keygen -t rsa -b 4096 -f ~/.ssh/thrillwiki_vm

# Copy public key to VM
ssh-copy-id -i ~/.ssh/thrillwiki_vm.pub ubuntu@VM_IP_ADDRESS
```

### 1.4 Clone Repository

```bash
# On the VM
cd /home/ubuntu
git clone https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git thrillwiki
cd thrillwiki
```

### 1.5 Install Dependencies

```bash
# Install Python dependencies
uv sync

# Create required directories
mkdir -p logs backups
```

## Step 2: Configure Services

### 2.1 Install Systemd Services

```bash
# Copy service files to systemd directory
sudo cp scripts/systemd/thrillwiki.service /etc/systemd/system/
sudo cp scripts/systemd/thrillwiki-webhook.service /etc/systemd/system/

# Edit service files to match your paths
sudo nano /etc/systemd/system/thrillwiki.service
sudo nano /etc/systemd/system/thrillwiki-webhook.service

# Reload systemd and enable services
sudo systemctl daemon-reload
sudo systemctl enable thrillwiki.service
sudo systemctl enable thrillwiki-webhook.service
```

### 2.2 Configure Environment Variables

Create `/home/ubuntu/thrillwiki/***REMOVED***`:

```bash
# Database configuration
DATABASE_URL=[DATABASE-URL-REMOVED]

# Django settings
DJANGO_SECRET_KEY=your_secret_key_here
DJANGO_DEBUG=False
DJANGO_ALLOWED_HOSTS=your_domain.com,VM_IP_ADDRESS

# Webhook configuration
WEBHOOK_SECRET=your_github_webhook_secret
WEBHOOK_PORT=9000
VM_HOST=localhost
VM_USER=ubuntu
VM_PROJECT_PATH=/home/ubuntu/thrillwiki
REPO_URL=https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git
```

## Step 3: Local Machine Setup

### 3.1 Configure Webhook Listener

Create a configuration file for the webhook listener:

```bash
# Create environment file
cat > ***REMOVED***.webhook << EOF
WEBHOOK_PORT=9000
WEBHOOK_SECRET=your_github_webhook_secret
VM_HOST=VM_IP_ADDRESS
VM_PORT=22
VM_USER=ubuntu
VM_KEY_PATH=/home/your_user/.ssh/thrillwiki_vm
VM_PROJECT_PATH=/home/ubuntu/thrillwiki
REPO_URL=https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git
DEPLOY_BRANCH=main
EOF
```

### 3.2 Set up GitHub Webhook

1. Go to your GitHub repository
2. Navigate to Settings → Webhooks
3. Click "Add webhook"
4. Configure:
   - **Payload URL**: `http://YOUR_PUBLIC_IP:9000/webhook`
   - **Content type**: `application/json`
   - **Secret**: Your webhook secret
   - **Events**: Select "Just the push event"

## Step 4: Database Setup

### 4.1 PostgreSQL Configuration

```bash
# On the VM
sudo -u postgres psql

-- Create database and user
CREATE DATABASE thrillwiki;
CREATE USER thrillwiki_user WITH ENCRYPTED PASSWORD 'your_password';
GRANT ALL PRIVILEGES ON DATABASE thrillwiki TO thrillwiki_user;
\q

# Install PostGIS (if using geographic features)
sudo apt install -y postgresql-postgis postgresql-postgis-scripts
sudo -u postgres psql -d thrillwiki -c "CREATE EXTENSION postgis;"
```

### 4.2 Run Initial Migration

```bash
# On the VM
cd /home/ubuntu/thrillwiki
uv run manage.py migrate
uv run manage.py collectstatic --noinput
uv run manage.py createsuperuser
```

## Step 5: Start Services

### 5.1 Start VM Services

```bash
# On the VM
sudo systemctl start thrillwiki
sudo systemctl start thrillwiki-webhook
sudo systemctl status thrillwiki
sudo systemctl status thrillwiki-webhook
```

### 5.2 Start Local Webhook Listener

```bash
# On local machine
source ***REMOVED***.webhook
python3 scripts/webhook-listener.py
```

## Step 6: Testing

### 6.1 Test Local Server

```bash
# Start local development server
./scripts/ci-start.sh

# Check if server is running
curl http://localhost:8000/health
```

### 6.2 Test VM Deployment

```bash
# On the VM, test deployment script
./scripts/vm-deploy.sh

# Check service status
./scripts/vm-deploy.sh status

# View logs
journalctl -u thrillwiki -f
```

### 6.3 Test Webhook

```bash
# Test webhook endpoint
curl -X GET http://localhost:9000/health

# Make a test commit and push to trigger deployment
git add .
git commit -m "Test deployment"
git push origin main
```

## Monitoring and Logs

### Service Logs

```bash
# View service logs
journalctl -u thrillwiki -f
journalctl -u thrillwiki-webhook -f

# View deployment logs
tail -f /home/ubuntu/thrillwiki/logs/deploy.log
tail -f /home/ubuntu/thrillwiki/logs/webhook.log
```

### Health Checks

```bash
# Check services status
systemctl status thrillwiki
systemctl status thrillwiki-webhook

# Manual health check
curl http://localhost:8000/health
curl http://localhost:9000/health
```

## Troubleshooting

### Common Issues

1. **Permission Denied**
   ```bash
   # Fix file permissions
   chmod +x scripts/*.sh
   chown ubuntu:ubuntu -R /home/ubuntu/thrillwiki
   ```

2. **Service Won't Start**
   ```bash
   # Check service logs
   journalctl -u thrillwiki --no-pager
   
   # Verify paths in service files
   sudo systemctl edit thrillwiki
   ```

3. **Webhook Not Triggering**
   ```bash
   # Check webhook listener logs
   tail -f logs/webhook.log
   
   # Verify GitHub webhook configuration
   # Check firewall settings for port 9000
   ```

4. **Database Connection Issues**
   ```bash
   # Test database connection
   uv run manage.py dbshell
   
   # Check PostgreSQL status
   sudo systemctl status postgresql
   ```

### Rollback Procedure

If deployment fails, you can rollback:

```bash
# On the VM
./scripts/vm-deploy.sh
# The script automatically handles rollback on failure

# Manual rollback to specific commit
cd /home/ubuntu/thrillwiki
git reset --hard COMMIT_HASH
./scripts/vm-deploy.sh restart
```

## Security Considerations

1. **SSH Keys**: Use dedicated SSH keys for deployment
2. **Webhook Secret**: Use a strong, unique webhook secret
3. **Firewall**: Only open necessary ports (22, 8000, 9000)
4. **User Permissions**: Use dedicated deployment user with minimal privileges
5. **Environment Variables**: Store sensitive data in environment files, not in code

## Maintenance

### Regular Tasks

1. **Update Dependencies**: Run `uv sync` regularly
2. **Log Rotation**: Set up logrotate for application logs
3. **Backup Database**: Schedule regular database backups
4. **Monitor Disk Space**: Ensure sufficient space for logs and backups

### Cleanup Old Backups

```bash
# The deployment script automatically cleans old backups
# Manual cleanup if needed:
find /home/ubuntu/thrillwiki/backups -name "backup_*.commit" -mtime +30 -delete
```

## Performance Optimization

1. **Use Production WSGI Server**: Consider using Gunicorn instead of development server
2. **Reverse Proxy**: Set up Nginx as reverse proxy
3. **Database Optimization**: Configure PostgreSQL for production
4. **Static Files**: Serve static files through Nginx

This setup provides a robust CI/CD pipeline for automatic deployment of ThrillWiki to your Linux VM whenever code is pushed to GitHub.