mirror of
https://github.com/pacnpal/thrillwiki_django_no_react.git
synced 2025-12-20 08:31:08 -05:00
feat: complete monorepo structure with frontend and shared resources
- Add complete backend/ directory with full Django application - Add frontend/ directory with Vite + TypeScript setup ready for Next.js - Add comprehensive shared/ directory with: - Complete documentation and memory-bank archives - Media files and avatars (letters, park/ride images) - Deployment scripts and automation tools - Shared types and utilities - Add architecture/ directory with migration guides - Configure pnpm workspace for monorepo development - Update .gitignore to exclude .django_tailwind_cli/ build artifacts - Preserve all historical documentation in shared/docs/memory-bank/ - Set up proper structure for full-stack development with shared resources
This commit is contained in:
pacnpal
359
shared/docs/VM_DEPLOYMENT_SETUP.md
Normal file
359
shared/docs/VM_DEPLOYMENT_SETUP.md
Normal file
@@ -0,0 +1,359 @@
|
||||
# ThrillWiki VM Deployment Setup Guide
|
||||
|
||||
This guide explains how to set up a local CI/CD system that automatically deploys ThrillWiki to a Linux VM when commits are pushed to GitHub.
|
||||
|
||||
## System Overview
|
||||
|
||||
The deployment system consists of three main components:
|
||||
|
||||
1. **Local CI Start Script** (`scripts/ci-start.sh`) - Starts the Django server locally
|
||||
2. **GitHub Webhook Listener** (`scripts/webhook-listener.py`) - Listens for GitHub push events
|
||||
3. **VM Deployment Script** (`scripts/vm-deploy.sh`) - Deploys code changes to the Linux VM
|
||||
|
||||
## Architecture Flow
|
||||
|
||||
```
|
||||
GitHub Push → Webhook → Local Listener → SSH to VM → Deploy Script → Restart Server
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Local Machine (Webhook Listener Host)
|
||||
- Python 3.8+
|
||||
- SSH access to the Linux VM
|
||||
- Git repository with webhook access
|
||||
|
||||
### Linux VM (Deployment Target)
|
||||
- Ubuntu 20.04+ (recommended)
|
||||
- Python 3.8+
|
||||
- UV package manager
|
||||
- Git
|
||||
- PostgreSQL (if using database)
|
||||
- SSH server running
|
||||
- Sudo access for the deployment user
|
||||
|
||||
## Step 1: Linux VM Setup
|
||||
|
||||
### 1.1 Create Deployment User
|
||||
|
||||
```bash
|
||||
# On the Linux VM
|
||||
sudo adduser ubuntu
|
||||
sudo usermod -aG sudo ubuntu
|
||||
su - ubuntu
|
||||
```
|
||||
|
||||
### 1.2 Install Required Software
|
||||
|
||||
```bash
|
||||
# Update system
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Install essential packages
|
||||
sudo apt install -y git curl build-essential python3-pip python3-venv postgresql postgresql-contrib nginx
|
||||
|
||||
# Install UV package manager
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
source ~/.cargo/env
|
||||
```
|
||||
|
||||
### 1.3 Set up SSH Keys
|
||||
|
||||
```bash
|
||||
# Generate SSH key on local machine
|
||||
ssh-keygen -t rsa -b 4096 -f ~/.ssh/thrillwiki_vm
|
||||
|
||||
# Copy public key to VM
|
||||
ssh-copy-id -i ~/.ssh/thrillwiki_vm.pub ubuntu@VM_IP_ADDRESS
|
||||
```
|
||||
|
||||
### 1.4 Clone Repository
|
||||
|
||||
```bash
|
||||
# On the VM
|
||||
cd /home/ubuntu
|
||||
git clone https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git thrillwiki
|
||||
cd thrillwiki
|
||||
```
|
||||
|
||||
### 1.5 Install Dependencies
|
||||
|
||||
```bash
|
||||
# Install Python dependencies
|
||||
uv sync
|
||||
|
||||
# Create required directories
|
||||
mkdir -p logs backups
|
||||
```
|
||||
|
||||
## Step 2: Configure Services
|
||||
|
||||
### 2.1 Install Systemd Services
|
||||
|
||||
```bash
|
||||
# Copy service files to systemd directory
|
||||
sudo cp scripts/systemd/thrillwiki.service /etc/systemd/system/
|
||||
sudo cp scripts/systemd/thrillwiki-webhook.service /etc/systemd/system/
|
||||
|
||||
# Edit service files to match your paths
|
||||
sudo nano /etc/systemd/system/thrillwiki.service
|
||||
sudo nano /etc/systemd/system/thrillwiki-webhook.service
|
||||
|
||||
# Reload systemd and enable services
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable thrillwiki.service
|
||||
sudo systemctl enable thrillwiki-webhook.service
|
||||
```
|
||||
|
||||
### 2.2 Configure Environment Variables
|
||||
|
||||
Create `/home/ubuntu/thrillwiki/***REMOVED***`:
|
||||
|
||||
```bash
|
||||
# Database configuration
|
||||
DATABASE_URL=[DATABASE-URL-REMOVED]
|
||||
|
||||
# Django settings
|
||||
DJANGO_SECRET_KEY=your_secret_key_here
|
||||
DJANGO_DEBUG=False
|
||||
DJANGO_ALLOWED_HOSTS=your_domain.com,VM_IP_ADDRESS
|
||||
|
||||
# Webhook configuration
|
||||
WEBHOOK_SECRET=your_github_webhook_secret
|
||||
WEBHOOK_PORT=9000
|
||||
VM_HOST=localhost
|
||||
VM_USER=ubuntu
|
||||
VM_PROJECT_PATH=/home/ubuntu/thrillwiki
|
||||
REPO_URL=https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git
|
||||
```
|
||||
|
||||
## Step 3: Local Machine Setup
|
||||
|
||||
### 3.1 Configure Webhook Listener
|
||||
|
||||
Create a configuration file for the webhook listener:
|
||||
|
||||
```bash
|
||||
# Create environment file
|
||||
cat > ***REMOVED***.webhook << EOF
|
||||
WEBHOOK_PORT=9000
|
||||
WEBHOOK_SECRET=your_github_webhook_secret
|
||||
VM_HOST=VM_IP_ADDRESS
|
||||
VM_PORT=22
|
||||
VM_USER=ubuntu
|
||||
VM_KEY_PATH=/home/your_user/.ssh/thrillwiki_vm
|
||||
VM_PROJECT_PATH=/home/ubuntu/thrillwiki
|
||||
REPO_URL=https://github.com/YOUR_USERNAME/thrillwiki_django_no_react.git
|
||||
DEPLOY_BRANCH=main
|
||||
EOF
|
||||
```
|
||||
|
||||
### 3.2 Set up GitHub Webhook
|
||||
|
||||
1. Go to your GitHub repository
|
||||
2. Navigate to Settings → Webhooks
|
||||
3. Click "Add webhook"
|
||||
4. Configure:
|
||||
- **Payload URL**: `http://YOUR_PUBLIC_IP:9000/webhook`
|
||||
- **Content type**: `application/json`
|
||||
- **Secret**: Your webhook secret
|
||||
- **Events**: Select "Just the push event"
|
||||
|
||||
## Step 4: Database Setup
|
||||
|
||||
### 4.1 PostgreSQL Configuration
|
||||
|
||||
```bash
|
||||
# On the VM
|
||||
sudo -u postgres psql
|
||||
|
||||
-- Create database and user
|
||||
CREATE DATABASE thrillwiki;
|
||||
CREATE USER thrillwiki_user WITH ENCRYPTED PASSWORD 'your_password';
|
||||
GRANT ALL PRIVILEGES ON DATABASE thrillwiki TO thrillwiki_user;
|
||||
\q
|
||||
|
||||
# Install PostGIS (if using geographic features)
|
||||
sudo apt install -y postgresql-postgis postgresql-postgis-scripts
|
||||
sudo -u postgres psql -d thrillwiki -c "CREATE EXTENSION postgis;"
|
||||
```
|
||||
|
||||
### 4.2 Run Initial Migration
|
||||
|
||||
```bash
|
||||
# On the VM
|
||||
cd /home/ubuntu/thrillwiki
|
||||
uv run manage.py migrate
|
||||
uv run manage.py collectstatic --noinput
|
||||
uv run manage.py createsuperuser
|
||||
```
|
||||
|
||||
## Step 5: Start Services
|
||||
|
||||
### 5.1 Start VM Services
|
||||
|
||||
```bash
|
||||
# On the VM
|
||||
sudo systemctl start thrillwiki
|
||||
sudo systemctl start thrillwiki-webhook
|
||||
sudo systemctl status thrillwiki
|
||||
sudo systemctl status thrillwiki-webhook
|
||||
```
|
||||
|
||||
### 5.2 Start Local Webhook Listener
|
||||
|
||||
```bash
|
||||
# On local machine
|
||||
source ***REMOVED***.webhook
|
||||
python3 scripts/webhook-listener.py
|
||||
```
|
||||
|
||||
## Step 6: Testing
|
||||
|
||||
### 6.1 Test Local Server
|
||||
|
||||
```bash
|
||||
# Start local development server
|
||||
./scripts/ci-start.sh
|
||||
|
||||
# Check if server is running
|
||||
curl http://localhost:8000/health
|
||||
```
|
||||
|
||||
### 6.2 Test VM Deployment
|
||||
|
||||
```bash
|
||||
# On the VM, test deployment script
|
||||
./scripts/vm-deploy.sh
|
||||
|
||||
# Check service status
|
||||
./scripts/vm-deploy.sh status
|
||||
|
||||
# View logs
|
||||
journalctl -u thrillwiki -f
|
||||
```
|
||||
|
||||
### 6.3 Test Webhook
|
||||
|
||||
```bash
|
||||
# Test webhook endpoint
|
||||
curl -X GET http://localhost:9000/health
|
||||
|
||||
# Make a test commit and push to trigger deployment
|
||||
git add .
|
||||
git commit -m "Test deployment"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
## Monitoring and Logs
|
||||
|
||||
### Service Logs
|
||||
|
||||
```bash
|
||||
# View service logs
|
||||
journalctl -u thrillwiki -f
|
||||
journalctl -u thrillwiki-webhook -f
|
||||
|
||||
# View deployment logs
|
||||
tail -f /home/ubuntu/thrillwiki/logs/deploy.log
|
||||
tail -f /home/ubuntu/thrillwiki/logs/webhook.log
|
||||
```
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Check services status
|
||||
systemctl status thrillwiki
|
||||
systemctl status thrillwiki-webhook
|
||||
|
||||
# Manual health check
|
||||
curl http://localhost:8000/health
|
||||
curl http://localhost:9000/health
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Permission Denied**
|
||||
```bash
|
||||
# Fix file permissions
|
||||
chmod +x scripts/*.sh
|
||||
chown ubuntu:ubuntu -R /home/ubuntu/thrillwiki
|
||||
```
|
||||
|
||||
2. **Service Won't Start**
|
||||
```bash
|
||||
# Check service logs
|
||||
journalctl -u thrillwiki --no-pager
|
||||
|
||||
# Verify paths in service files
|
||||
sudo systemctl edit thrillwiki
|
||||
```
|
||||
|
||||
3. **Webhook Not Triggering**
|
||||
```bash
|
||||
# Check webhook listener logs
|
||||
tail -f logs/webhook.log
|
||||
|
||||
# Verify GitHub webhook configuration
|
||||
# Check firewall settings for port 9000
|
||||
```
|
||||
|
||||
4. **Database Connection Issues**
|
||||
```bash
|
||||
# Test database connection
|
||||
uv run manage.py dbshell
|
||||
|
||||
# Check PostgreSQL status
|
||||
sudo systemctl status postgresql
|
||||
```
|
||||
|
||||
### Rollback Procedure
|
||||
|
||||
If deployment fails, you can rollback:
|
||||
|
||||
```bash
|
||||
# On the VM
|
||||
./scripts/vm-deploy.sh
|
||||
# The script automatically handles rollback on failure
|
||||
|
||||
# Manual rollback to specific commit
|
||||
cd /home/ubuntu/thrillwiki
|
||||
git reset --hard COMMIT_HASH
|
||||
./scripts/vm-deploy.sh restart
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **SSH Keys**: Use dedicated SSH keys for deployment
|
||||
2. **Webhook Secret**: Use a strong, unique webhook secret
|
||||
3. **Firewall**: Only open necessary ports (22, 8000, 9000)
|
||||
4. **User Permissions**: Use dedicated deployment user with minimal privileges
|
||||
5. **Environment Variables**: Store sensitive data in environment files, not in code
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Regular Tasks
|
||||
|
||||
1. **Update Dependencies**: Run `uv sync` regularly
|
||||
2. **Log Rotation**: Set up logrotate for application logs
|
||||
3. **Backup Database**: Schedule regular database backups
|
||||
4. **Monitor Disk Space**: Ensure sufficient space for logs and backups
|
||||
|
||||
### Cleanup Old Backups
|
||||
|
||||
```bash
|
||||
# The deployment script automatically cleans old backups
|
||||
# Manual cleanup if needed:
|
||||
find /home/ubuntu/thrillwiki/backups -name "backup_*.commit" -mtime +30 -delete
|
||||
```
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
1. **Use Production WSGI Server**: Consider using Gunicorn instead of development server
|
||||
2. **Reverse Proxy**: Set up Nginx as reverse proxy
|
||||
3. **Database Optimization**: Configure PostgreSQL for production
|
||||
4. **Static Files**: Serve static files through Nginx
|
||||
|
||||
This setup provides a robust CI/CD pipeline for automatic deployment of ThrillWiki to your Linux VM whenever code is pushed to GitHub.
|
||||
Reference in New Issue
Block a user