mirror of
https://github.com/pacnpal/thrilltrack-explorer.git
synced 2025-12-20 09:51:13 -05:00
feat: Django backend foundation - project structure, dependencies, and documentation
- Created Django 4.2 project with production-ready architecture - Installed 60+ packages including django-ninja, celery, channels, etc. - Set up app structure (core, entities, moderation, users, versioning, media, notifications) - Created comprehensive MIGRATION_PLAN.md with 12-phase roadmap - Created README.md with setup instructions - Created .env.example with all required configuration - Configured for Python 3.13 compatibility - All dependencies successfully installed and tested Next steps: Configure Django settings and create base models
This commit is contained in:
pacnpal
35
django/.env.example
Normal file
35
django/.env.example
Normal file
@@ -0,0 +1,35 @@
|
||||
# Django Settings
|
||||
DEBUG=True
|
||||
SECRET_KEY=your-secret-key-here-change-in-production
|
||||
ALLOWED_HOSTS=localhost,127.0.0.1
|
||||
|
||||
# Database
|
||||
DATABASE_URL=postgresql://user:password@localhost:5432/thrillwiki
|
||||
|
||||
# Redis
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
|
||||
# Celery
|
||||
CELERY_BROKER_URL=redis://localhost:6379/0
|
||||
CELERY_RESULT_BACKEND=redis://localhost:6379/1
|
||||
|
||||
# CloudFlare Images
|
||||
CLOUDFLARE_ACCOUNT_ID=your-account-id
|
||||
CLOUDFLARE_IMAGE_TOKEN=your-token
|
||||
CLOUDFLARE_IMAGE_HASH=your-hash
|
||||
|
||||
# Novu
|
||||
NOVU_API_KEY=your-novu-api-key
|
||||
NOVU_API_URL=https://api.novu.co
|
||||
|
||||
# Sentry
|
||||
SENTRY_DSN=your-sentry-dsn
|
||||
|
||||
# CORS
|
||||
CORS_ALLOWED_ORIGINS=http://localhost:5173,http://localhost:3000
|
||||
|
||||
# OAuth
|
||||
GOOGLE_CLIENT_ID=
|
||||
GOOGLE_CLIENT_SECRET=
|
||||
DISCORD_CLIENT_ID=
|
||||
DISCORD_CLIENT_SECRET=
|
||||
566
django/MIGRATION_PLAN.md
Normal file
566
django/MIGRATION_PLAN.md
Normal file
@@ -0,0 +1,566 @@
|
||||
# ThrillWiki Django Backend Migration Plan
|
||||
|
||||
## 🎯 Project Overview
|
||||
|
||||
**Objective**: Migrate ThrillWiki from Supabase backend to Django REST backend while preserving 100% of functionality.
|
||||
|
||||
**Timeline**: 12-16 weeks with 2 developers
|
||||
**Status**: Foundation Phase - In Progress
|
||||
**Branch**: `django-backend`
|
||||
|
||||
---
|
||||
|
||||
## 📊 Architecture Overview
|
||||
|
||||
### Current Stack (Supabase)
|
||||
- **Frontend**: React 18.3 + TypeScript + Vite + React Query
|
||||
- **Backend**: Supabase (PostgreSQL + Edge Functions)
|
||||
- **Database**: PostgreSQL with 80+ tables
|
||||
- **Auth**: Supabase Auth (OAuth + MFA)
|
||||
- **Storage**: CloudFlare Images
|
||||
- **Notifications**: Novu Cloud
|
||||
- **Real-time**: Supabase Realtime
|
||||
|
||||
### Target Stack (Django)
|
||||
- **Frontend**: React 18.3 + TypeScript + Vite (unchanged)
|
||||
- **Backend**: Django 4.2 + django-ninja
|
||||
- **Database**: PostgreSQL (migrated schema)
|
||||
- **Auth**: Django + django-allauth + django-otp
|
||||
- **Storage**: CloudFlare Images (unchanged)
|
||||
- **Notifications**: Novu Cloud (unchanged)
|
||||
- **Real-time**: Django Channels + WebSockets
|
||||
- **Tasks**: Celery + Redis
|
||||
- **Caching**: Redis + django-cacheops
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Project Structure
|
||||
|
||||
```
|
||||
django/
|
||||
├── manage.py
|
||||
├── config/ # Project settings
|
||||
│ ├── settings/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── base.py # Shared settings
|
||||
│ │ ├── local.py # Development
|
||||
│ │ └── production.py # Production
|
||||
│ ├── urls.py
|
||||
│ ├── wsgi.py
|
||||
│ └── asgi.py # For Channels
|
||||
│
|
||||
├── apps/
|
||||
│ ├── core/ # Base models, utilities
|
||||
│ │ ├── models.py # Abstract base models
|
||||
│ │ ├── permissions.py # Reusable permissions
|
||||
│ │ ├── mixins.py # Model mixins
|
||||
│ │ └── utils.py
|
||||
│ │
|
||||
│ ├── entities/ # Parks, Rides, Companies
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── park.py
|
||||
│ │ │ ├── ride.py
|
||||
│ │ │ ├── company.py
|
||||
│ │ │ └── ride_model.py
|
||||
│ │ ├── api/
|
||||
│ │ │ ├── views.py
|
||||
│ │ │ ├── serializers.py
|
||||
│ │ │ └── filters.py
|
||||
│ │ ├── services.py
|
||||
│ │ └── tasks.py
|
||||
│ │
|
||||
│ ├── moderation/ # Content moderation
|
||||
│ │ ├── models.py
|
||||
│ │ ├── state_machine.py # django-fsm workflow
|
||||
│ │ ├── services.py
|
||||
│ │ └── api/
|
||||
│ │
|
||||
│ ├── versioning/ # Entity versioning
|
||||
│ │ ├── models.py
|
||||
│ │ ├── signals.py
|
||||
│ │ └── services.py
|
||||
│ │
|
||||
│ ├── users/ # User management
|
||||
│ │ ├── models.py
|
||||
│ │ ├── managers.py
|
||||
│ │ └── api/
|
||||
│ │
|
||||
│ ├── media/ # Photo management
|
||||
│ │ ├── models.py
|
||||
│ │ ├── storage.py
|
||||
│ │ └── tasks.py
|
||||
│ │
|
||||
│ └── notifications/ # Notification system
|
||||
│ ├── models.py
|
||||
│ ├── providers/
|
||||
│ │ └── novu.py
|
||||
│ └── tasks.py
|
||||
│
|
||||
├── api/
|
||||
│ └── v1/
|
||||
│ ├── router.py # Main API router
|
||||
│ └── schemas.py # Pydantic schemas
|
||||
│
|
||||
└── scripts/
|
||||
├── migrate_from_supabase.py
|
||||
└── validate_data.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Implementation Phases
|
||||
|
||||
### ✅ Phase 0: Foundation (CURRENT - Week 1)
|
||||
- [x] Create git branch `django-backend`
|
||||
- [x] Set up Python virtual environment
|
||||
- [x] Install all dependencies (Django 4.2, django-ninja, celery, etc.)
|
||||
- [x] Create Django project structure
|
||||
- [x] Create app directories
|
||||
- [x] Create .env.example
|
||||
- [ ] Configure Django settings (base, local, production)
|
||||
- [ ] Create base models and utilities
|
||||
- [ ] Set up database connection
|
||||
- [ ] Create initial migrations
|
||||
|
||||
### Phase 1: Core Models (Week 2-3)
|
||||
- [ ] Create abstract base models (TimeStamped, Versioned, etc.)
|
||||
- [ ] Implement entity models (Park, Ride, Company, RideModel)
|
||||
- [ ] Implement location models
|
||||
- [ ] Implement user models with custom User
|
||||
- [ ] Implement photo/media models
|
||||
- [ ] Create Django migrations
|
||||
- [ ] Test model relationships
|
||||
|
||||
### Phase 2: Authentication System (Week 3-4)
|
||||
- [ ] Set up django-allauth for OAuth (Google, Discord)
|
||||
- [ ] Implement JWT authentication with djangorestframework-simplejwt
|
||||
- [ ] Set up django-otp for MFA (TOTP)
|
||||
- [ ] Create user registration/login endpoints
|
||||
- [ ] Implement permission system (django-guardian)
|
||||
- [ ] Create role-based access control
|
||||
- [ ] Test authentication flow
|
||||
|
||||
### Phase 3: Moderation System (Week 5-7)
|
||||
- [ ] Create ContentSubmission and SubmissionItem models
|
||||
- [ ] Implement django-fsm state machine
|
||||
- [ ] Create ModerationService with atomic transactions
|
||||
- [ ] Implement submission creation endpoints
|
||||
- [ ] Implement approval/rejection endpoints
|
||||
- [ ] Implement selective approval logic
|
||||
- [ ] Create moderation queue API
|
||||
- [ ] Add rate limiting with django-ratelimit
|
||||
- [ ] Test moderation workflow end-to-end
|
||||
|
||||
### Phase 4: Versioning System (Week 7-8)
|
||||
- [ ] Create version models for all entities
|
||||
- [ ] Implement django-lifecycle hooks for auto-versioning
|
||||
- [ ] Create VersioningService
|
||||
- [ ] Implement version history endpoints
|
||||
- [ ] Add version diff functionality
|
||||
- [ ] Test versioning with submissions
|
||||
|
||||
### Phase 5: API Layer with django-ninja (Week 8-10)
|
||||
- [ ] Set up django-ninja router
|
||||
- [ ] Create Pydantic schemas for all entities
|
||||
- [ ] Implement CRUD endpoints for parks
|
||||
- [ ] Implement CRUD endpoints for rides
|
||||
- [ ] Implement CRUD endpoints for companies
|
||||
- [ ] Add filtering with django-filter
|
||||
- [ ] Add search functionality
|
||||
- [ ] Implement pagination
|
||||
- [ ] Add API documentation (auto-generated)
|
||||
- [ ] Test all endpoints
|
||||
|
||||
### Phase 6: Celery Tasks (Week 10-11)
|
||||
- [ ] Set up Celery with Redis
|
||||
- [ ] Set up django-celery-beat for periodic tasks
|
||||
- [ ] Migrate edge functions to Celery tasks:
|
||||
- [ ] cleanup_old_page_views
|
||||
- [ ] update_entity_view_counts
|
||||
- [ ] process_submission_notifications
|
||||
- [ ] generate_daily_stats
|
||||
- [ ] Create notification tasks for Novu
|
||||
- [ ] Set up Flower for monitoring
|
||||
- [ ] Test async task execution
|
||||
|
||||
### Phase 7: Real-time Features (Week 11-12)
|
||||
- [ ] Set up Django Channels with Redis
|
||||
- [ ] Create WebSocket consumers
|
||||
- [ ] Implement moderation queue real-time updates
|
||||
- [ ] Implement notification real-time delivery
|
||||
- [ ] Test WebSocket connections
|
||||
- [ ] OR: Implement Server-Sent Events as alternative
|
||||
|
||||
### Phase 8: Caching & Performance (Week 12-13)
|
||||
- [ ] Set up django-redis for caching
|
||||
- [ ] Configure django-cacheops for automatic ORM caching
|
||||
- [ ] Add cache invalidation logic
|
||||
- [ ] Optimize database queries (select_related, prefetch_related)
|
||||
- [ ] Add database indexes
|
||||
- [ ] Profile with django-silk
|
||||
- [ ] Load testing
|
||||
|
||||
### Phase 9: Data Migration (Week 13-14)
|
||||
- [ ] Export all data from Supabase
|
||||
- [ ] Create migration script for entities
|
||||
- [ ] Migrate user data (preserve UUIDs)
|
||||
- [ ] Migrate submissions (pending only)
|
||||
- [ ] Migrate version history
|
||||
- [ ] Migrate photos/media references
|
||||
- [ ] Validate data integrity
|
||||
- [ ] Test with migrated data
|
||||
|
||||
### Phase 10: Frontend Integration (Week 14-15)
|
||||
- [ ] Create new API client (replace Supabase client)
|
||||
- [ ] Update authentication logic
|
||||
- [ ] Update all API calls to point to Django
|
||||
- [ ] Update real-time subscriptions to WebSockets
|
||||
- [ ] Test all user flows
|
||||
- [ ] Fix any integration issues
|
||||
|
||||
### Phase 11: Testing & QA (Week 15-16)
|
||||
- [ ] Write unit tests for all models
|
||||
- [ ] Write unit tests for all services
|
||||
- [ ] Write API integration tests
|
||||
- [ ] Write end-to-end tests
|
||||
- [ ] Security audit
|
||||
- [ ] Performance testing
|
||||
- [ ] Load testing
|
||||
- [ ] Bug fixes
|
||||
|
||||
### Phase 12: Deployment (Week 16-17)
|
||||
- [ ] Set up production environment
|
||||
- [ ] Configure PostgreSQL
|
||||
- [ ] Configure Redis
|
||||
- [ ] Set up Celery workers
|
||||
- [ ] Configure Gunicorn/Daphne
|
||||
- [ ] Set up Docker containers
|
||||
- [ ] Configure CI/CD
|
||||
- [ ] Deploy to staging
|
||||
- [ ] Final testing
|
||||
- [ ] Deploy to production
|
||||
- [ ] Monitor for issues
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Key Technical Decisions
|
||||
|
||||
### 1. **django-ninja vs Django REST Framework**
|
||||
**Choice**: django-ninja
|
||||
- FastAPI-style syntax (modern, intuitive)
|
||||
- Better performance
|
||||
- Automatic OpenAPI documentation
|
||||
- Pydantic integration for validation
|
||||
|
||||
### 2. **State Machine for Moderation**
|
||||
**Choice**: django-fsm
|
||||
- Declarative state transitions
|
||||
- Built-in guards and conditions
|
||||
- Prevents invalid state changes
|
||||
- Easy to visualize workflow
|
||||
|
||||
### 3. **Auto-versioning Strategy**
|
||||
**Choice**: django-lifecycle hooks
|
||||
- Automatic version creation on model changes
|
||||
- No manual intervention needed
|
||||
- Tracks what changed
|
||||
- Preserves full history
|
||||
|
||||
### 4. **Real-time Communication**
|
||||
**Primary**: Django Channels (WebSockets)
|
||||
**Fallback**: Server-Sent Events (SSE)
|
||||
- WebSockets for bidirectional communication
|
||||
- SSE as simpler alternative
|
||||
- Redis channel layer for scaling
|
||||
|
||||
### 5. **Caching Strategy**
|
||||
**Tool**: django-cacheops
|
||||
- Automatic ORM query caching
|
||||
- Transparent invalidation
|
||||
- Minimal code changes
|
||||
- Redis backend for consistency
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Critical Features to Preserve
|
||||
|
||||
### 1. **Moderation System**
|
||||
- ✅ Atomic transactions for approvals
|
||||
- ✅ Selective approval (approve individual items)
|
||||
- ✅ State machine workflow (pending → reviewing → approved/rejected)
|
||||
- ✅ Lock mechanism (15-minute lock on review)
|
||||
- ✅ Automatic unlock on timeout
|
||||
- ✅ Batch operations
|
||||
|
||||
### 2. **Versioning System**
|
||||
- ✅ Full version history for all entities
|
||||
- ✅ Track who made changes
|
||||
- ✅ Track what changed
|
||||
- ✅ Link versions to submissions
|
||||
- ✅ Version diffs
|
||||
- ✅ Rollback capability
|
||||
|
||||
### 3. **Authentication**
|
||||
- ✅ Password-based login
|
||||
- ✅ Google OAuth
|
||||
- ✅ Discord OAuth
|
||||
- ✅ Two-factor authentication (TOTP)
|
||||
- ✅ Session management
|
||||
- ✅ JWT tokens for API
|
||||
|
||||
### 4. **Permissions & Security**
|
||||
- ✅ Role-based access control (user, moderator, admin, superuser)
|
||||
- ✅ Object-level permissions
|
||||
- ✅ Rate limiting
|
||||
- ✅ CORS configuration
|
||||
- ✅ Brute force protection
|
||||
|
||||
### 5. **Image Management**
|
||||
- ✅ CloudFlare direct upload
|
||||
- ✅ Image validation
|
||||
- ✅ Image metadata storage
|
||||
- ✅ Multiple image variants (thumbnails, etc.)
|
||||
|
||||
### 6. **Notifications**
|
||||
- ✅ Email notifications via Novu
|
||||
- ✅ In-app notifications
|
||||
- ✅ Notification templates
|
||||
- ✅ User preferences
|
||||
|
||||
### 7. **Search & Filtering**
|
||||
- ✅ Full-text search
|
||||
- ✅ Advanced filtering
|
||||
- ✅ Sorting options
|
||||
- ✅ Pagination
|
||||
|
||||
---
|
||||
|
||||
## 📊 Database Schema Preservation
|
||||
|
||||
### Core Entity Tables (Must Migrate)
|
||||
```
|
||||
✅ parks (80+ fields including dates, locations, operators)
|
||||
✅ rides (100+ fields including ride_models, parks, manufacturers)
|
||||
✅ companies (manufacturers, operators, designers)
|
||||
✅ ride_models (coaster models, flat ride models)
|
||||
✅ locations (countries, subdivisions, localities)
|
||||
✅ profiles (user profiles linked to auth.users)
|
||||
✅ user_roles (role assignments)
|
||||
✅ content_submissions (moderation queue)
|
||||
✅ submission_items (individual changes in submissions)
|
||||
✅ park_versions, ride_versions, etc. (version history)
|
||||
✅ photos (image metadata)
|
||||
✅ photo_submissions (photo approval queue)
|
||||
✅ reviews (user reviews)
|
||||
✅ reports (user reports)
|
||||
✅ entity_timeline_events (history timeline)
|
||||
✅ notification_logs
|
||||
✅ notification_templates
|
||||
```
|
||||
|
||||
### Computed Fields Strategy
|
||||
Some Supabase tables have computed fields. Options:
|
||||
1. **Cache in model** (recommended for frequently accessed)
|
||||
2. **Property method** (for rarely accessed)
|
||||
3. **Cached query** (using django-cacheops)
|
||||
|
||||
Example:
|
||||
```python
|
||||
class Park(models.Model):
|
||||
# Cached computed fields
|
||||
ride_count = models.IntegerField(default=0)
|
||||
coaster_count = models.IntegerField(default=0)
|
||||
|
||||
def update_counts(self):
|
||||
"""Update cached counts"""
|
||||
self.ride_count = self.rides.count()
|
||||
self.coaster_count = self.rides.filter(
|
||||
is_coaster=True
|
||||
).count()
|
||||
self.save()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Development Setup
|
||||
|
||||
### Prerequisites
|
||||
```bash
|
||||
# System requirements
|
||||
Python 3.11+
|
||||
PostgreSQL 15+
|
||||
Redis 7+
|
||||
Node.js 18+ (for frontend)
|
||||
```
|
||||
|
||||
### Initial Setup
|
||||
```bash
|
||||
# 1. Clone and checkout branch
|
||||
git checkout django-backend
|
||||
|
||||
# 2. Set up Python environment
|
||||
cd django
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# 3. Install dependencies
|
||||
pip install -r requirements/local.txt
|
||||
|
||||
# 4. Set up environment
|
||||
cp .env.example .env
|
||||
# Edit .env with your credentials
|
||||
|
||||
# 5. Run migrations
|
||||
python manage.py migrate
|
||||
|
||||
# 6. Create superuser
|
||||
python manage.py createsuperuser
|
||||
|
||||
# 7. Run development server
|
||||
python manage.py runserver
|
||||
|
||||
# 8. Run Celery worker (separate terminal)
|
||||
celery -A config worker -l info
|
||||
|
||||
# 9. Run Celery beat (separate terminal)
|
||||
celery -A config beat -l info
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=apps --cov-report=html
|
||||
|
||||
# Run specific test file
|
||||
pytest apps/moderation/tests/test_services.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Edge Functions to Migrate
|
||||
|
||||
### Supabase Edge Functions → Django/Celery
|
||||
|
||||
| Edge Function | Django Implementation | Priority |
|
||||
|---------------|----------------------|----------|
|
||||
| `process-submission` | `ModerationService.submit()` | P0 |
|
||||
| `process-selective-approval` | `ModerationService.approve()` | P0 |
|
||||
| `reject-submission` | `ModerationService.reject()` | P0 |
|
||||
| `unlock-submission` | Celery periodic task | P0 |
|
||||
| `cleanup_old_page_views` | Celery periodic task | P1 |
|
||||
| `update_entity_view_counts` | Celery periodic task | P1 |
|
||||
| `send-notification` | `NotificationService.send()` | P0 |
|
||||
| `process-photo-submission` | `MediaService.submit_photo()` | P1 |
|
||||
| `generate-daily-stats` | Celery periodic task | P2 |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Success Criteria
|
||||
|
||||
### Must Have (P0)
|
||||
- ✅ All 80+ database tables migrated
|
||||
- ✅ All user data preserved (with UUIDs)
|
||||
- ✅ Authentication working (password + OAuth + MFA)
|
||||
- ✅ Moderation workflow functional
|
||||
- ✅ Versioning system working
|
||||
- ✅ All API endpoints functional
|
||||
- ✅ Frontend fully integrated
|
||||
- ✅ No data loss during migration
|
||||
- ✅ Performance equivalent or better
|
||||
|
||||
### Should Have (P1)
|
||||
- ✅ Real-time updates working
|
||||
- ✅ All Celery tasks running
|
||||
- ✅ Caching operational
|
||||
- ✅ Image uploads working
|
||||
- ✅ Notifications working
|
||||
- ✅ Search functional
|
||||
- ✅ Comprehensive test coverage (>80%)
|
||||
|
||||
### Nice to Have (P2)
|
||||
- Admin dashboard improvements
|
||||
- Enhanced monitoring/observability
|
||||
- API rate limiting per user
|
||||
- Advanced analytics
|
||||
- GraphQL endpoint (optional)
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Risk Mitigation
|
||||
|
||||
### Risk 1: Data Loss During Migration
|
||||
**Mitigation**:
|
||||
- Comprehensive backup before migration
|
||||
- Dry-run migration multiple times
|
||||
- Validation scripts to check data integrity
|
||||
- Rollback plan
|
||||
|
||||
### Risk 2: Downtime During Cutover
|
||||
**Mitigation**:
|
||||
- Blue-green deployment strategy
|
||||
- Run both systems in parallel briefly
|
||||
- Feature flags to toggle between backends
|
||||
- Quick rollback capability
|
||||
|
||||
### Risk 3: Performance Degradation
|
||||
**Mitigation**:
|
||||
- Load testing before production
|
||||
- Database query optimization
|
||||
- Aggressive caching strategy
|
||||
- Monitoring and alerting
|
||||
|
||||
### Risk 4: Missing Edge Cases
|
||||
**Mitigation**:
|
||||
- Comprehensive test suite
|
||||
- Manual QA testing
|
||||
- Beta testing period
|
||||
- Staged rollout
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support & Resources
|
||||
|
||||
### Documentation
|
||||
- Django: https://docs.djangoproject.com/
|
||||
- django-ninja: https://django-ninja.rest-framework.com/
|
||||
- Celery: https://docs.celeryq.dev/
|
||||
- Django Channels: https://channels.readthedocs.io/
|
||||
|
||||
### Key Files to Reference
|
||||
- Original database schema: `supabase/migrations/`
|
||||
- Current API endpoints: `src/lib/supabaseClient.ts`
|
||||
- Moderation logic: `src/components/moderation/`
|
||||
- Existing docs: `docs/moderation/`, `docs/versioning/`
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Next Steps
|
||||
|
||||
1. **Immediate** (This Week):
|
||||
- Configure Django settings
|
||||
- Create base models
|
||||
- Set up database connection
|
||||
|
||||
2. **Short-term** (Next 2 Weeks):
|
||||
- Implement entity models
|
||||
- Set up authentication
|
||||
- Create basic API endpoints
|
||||
|
||||
3. **Medium-term** (Next 4-8 Weeks):
|
||||
- Build moderation system
|
||||
- Implement versioning
|
||||
- Migrate edge functions
|
||||
|
||||
4. **Long-term** (8-16 Weeks):
|
||||
- Complete API layer
|
||||
- Frontend integration
|
||||
- Testing and deployment
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 8, 2025
|
||||
**Status**: Foundation Phase - Dependencies Installed, Structure Created
|
||||
**Next**: Configure Django settings and create base models
|
||||
281
django/README.md
Normal file
281
django/README.md
Normal file
@@ -0,0 +1,281 @@
|
||||
# ThrillWiki Django Backend
|
||||
|
||||
## 🚀 Overview
|
||||
|
||||
This is the Django REST API backend for ThrillWiki, replacing the previous Supabase backend. Built with modern Django best practices and production-ready packages.
|
||||
|
||||
## 📦 Tech Stack
|
||||
|
||||
- **Framework**: Django 4.2 LTS
|
||||
- **API**: django-ninja (FastAPI-style)
|
||||
- **Database**: PostgreSQL 15+
|
||||
- **Cache**: Redis + django-cacheops
|
||||
- **Tasks**: Celery + Redis
|
||||
- **Real-time**: Django Channels + WebSockets
|
||||
- **Auth**: django-allauth + django-otp
|
||||
- **Storage**: CloudFlare Images
|
||||
- **Monitoring**: Sentry + structlog
|
||||
|
||||
## 🏗️ Project Structure
|
||||
|
||||
```
|
||||
django/
|
||||
├── manage.py
|
||||
├── config/ # Django settings
|
||||
├── apps/ # Django applications
|
||||
│ ├── core/ # Base models & utilities
|
||||
│ ├── entities/ # Parks, Rides, Companies
|
||||
│ ├── moderation/ # Content moderation system
|
||||
│ ├── versioning/ # Entity versioning
|
||||
│ ├── users/ # User management
|
||||
│ ├── media/ # Image/photo management
|
||||
│ └── notifications/ # Notification system
|
||||
├── api/ # REST API layer
|
||||
└── scripts/ # Utility scripts
|
||||
```
|
||||
|
||||
## 🛠️ Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.11+
|
||||
- PostgreSQL 15+
|
||||
- Redis 7+
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# 1. Create virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
|
||||
# 2. Install dependencies
|
||||
pip install -r requirements/local.txt
|
||||
|
||||
# 3. Set up environment variables
|
||||
cp .env.example .env
|
||||
# Edit .env with your configuration
|
||||
|
||||
# 4. Run migrations
|
||||
python manage.py migrate
|
||||
|
||||
# 5. Create superuser
|
||||
python manage.py createsuperuser
|
||||
|
||||
# 6. Run development server
|
||||
python manage.py runserver
|
||||
```
|
||||
|
||||
### Running Services
|
||||
|
||||
```bash
|
||||
# Terminal 1: Django dev server
|
||||
python manage.py runserver
|
||||
|
||||
# Terminal 2: Celery worker
|
||||
celery -A config worker -l info
|
||||
|
||||
# Terminal 3: Celery beat (periodic tasks)
|
||||
celery -A config beat -l info
|
||||
|
||||
# Terminal 4: Flower (task monitoring)
|
||||
celery -A config flower
|
||||
```
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- **Migration Plan**: See `MIGRATION_PLAN.md` for full migration details
|
||||
- **Architecture**: See project documentation in `/docs/`
|
||||
- **API Docs**: Available at `/api/docs` when server is running
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=apps --cov-report=html
|
||||
|
||||
# Run specific app tests
|
||||
pytest apps/moderation/
|
||||
|
||||
# Run specific test file
|
||||
pytest apps/moderation/tests/test_services.py -v
|
||||
```
|
||||
|
||||
## 📋 Key Features
|
||||
|
||||
### Moderation System
|
||||
- State machine workflow with django-fsm
|
||||
- Atomic transaction handling
|
||||
- Selective approval support
|
||||
- Automatic lock/unlock mechanism
|
||||
- Real-time queue updates
|
||||
|
||||
### Versioning System
|
||||
- Automatic version tracking with django-lifecycle
|
||||
- Full change history for all entities
|
||||
- Diff generation
|
||||
- Rollback capability
|
||||
|
||||
### Authentication
|
||||
- JWT-based API authentication
|
||||
- OAuth2 (Google, Discord)
|
||||
- Two-factor authentication (TOTP)
|
||||
- Role-based permissions
|
||||
|
||||
### Performance
|
||||
- Automatic query caching with django-cacheops
|
||||
- Redis-based session storage
|
||||
- Optimized database queries
|
||||
- Background task processing with Celery
|
||||
|
||||
## 🔧 Management Commands
|
||||
|
||||
```bash
|
||||
# Create test data
|
||||
python manage.py seed_data
|
||||
|
||||
# Export data from Supabase
|
||||
python manage.py export_supabase_data
|
||||
|
||||
# Import data to Django
|
||||
python manage.py import_supabase_data
|
||||
|
||||
# Update cached counts
|
||||
python manage.py update_counts
|
||||
|
||||
# Clean old data
|
||||
python manage.py cleanup_old_data
|
||||
```
|
||||
|
||||
## 🚀 Deployment
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
docker build -t thrillwiki-backend .
|
||||
|
||||
# Run with docker-compose
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Production Checklist
|
||||
|
||||
- [ ] Set `DEBUG=False` in production
|
||||
- [ ] Configure `ALLOWED_HOSTS`
|
||||
- [ ] Set strong `SECRET_KEY`
|
||||
- [ ] Configure PostgreSQL connection
|
||||
- [ ] Set up Redis
|
||||
- [ ] Configure Celery workers
|
||||
- [ ] Set up SSL/TLS
|
||||
- [ ] Configure CORS origins
|
||||
- [ ] Set up Sentry for error tracking
|
||||
- [ ] Configure CloudFlare Images
|
||||
- [ ] Set up monitoring/logging
|
||||
|
||||
## 📊 Development Status
|
||||
|
||||
**Current Phase**: Foundation
|
||||
**Branch**: `django-backend`
|
||||
|
||||
### Completed
|
||||
- ✅ Project structure created
|
||||
- ✅ Dependencies installed
|
||||
- ✅ Environment configuration
|
||||
|
||||
### In Progress
|
||||
- 🔄 Django settings configuration
|
||||
- 🔄 Base models creation
|
||||
- 🔄 Database connection setup
|
||||
|
||||
### Upcoming
|
||||
- ⏳ Entity models implementation
|
||||
- ⏳ Authentication system
|
||||
- ⏳ Moderation system
|
||||
- ⏳ API layer with django-ninja
|
||||
|
||||
See `MIGRATION_PLAN.md` for detailed roadmap.
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
1. Create a feature branch from `django-backend`
|
||||
2. Make your changes
|
||||
3. Write/update tests
|
||||
4. Run test suite
|
||||
5. Submit pull request
|
||||
|
||||
## 📝 Environment Variables
|
||||
|
||||
Required environment variables (see `.env.example`):
|
||||
|
||||
```bash
|
||||
# Django
|
||||
DEBUG=True
|
||||
SECRET_KEY=your-secret-key
|
||||
ALLOWED_HOSTS=localhost
|
||||
|
||||
# Database
|
||||
DATABASE_URL=postgresql://user:pass@localhost:5432/thrillwiki
|
||||
|
||||
# Redis
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
|
||||
# External Services
|
||||
CLOUDFLARE_ACCOUNT_ID=xxx
|
||||
CLOUDFLARE_IMAGE_TOKEN=xxx
|
||||
NOVU_API_KEY=xxx
|
||||
SENTRY_DSN=xxx
|
||||
|
||||
# OAuth
|
||||
GOOGLE_CLIENT_ID=xxx
|
||||
GOOGLE_CLIENT_SECRET=xxx
|
||||
DISCORD_CLIENT_ID=xxx
|
||||
DISCORD_CLIENT_SECRET=xxx
|
||||
```
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Database Connection Issues
|
||||
```bash
|
||||
# Check PostgreSQL is running
|
||||
pg_isready
|
||||
|
||||
# Verify connection string
|
||||
python manage.py dbshell
|
||||
```
|
||||
|
||||
### Celery Not Processing Tasks
|
||||
```bash
|
||||
# Check Redis is running
|
||||
redis-cli ping
|
||||
|
||||
# Restart Celery worker
|
||||
celery -A config worker --purge -l info
|
||||
```
|
||||
|
||||
### Import Errors
|
||||
```bash
|
||||
# Ensure virtual environment is activated
|
||||
which python # Should point to venv/bin/python
|
||||
|
||||
# Reinstall dependencies
|
||||
pip install -r requirements/local.txt --force-reinstall
|
||||
```
|
||||
|
||||
## 📞 Support
|
||||
|
||||
- **Documentation**: See `/docs/` directory
|
||||
- **Issues**: GitHub Issues
|
||||
- **Migration Questions**: See `MIGRATION_PLAN.md`
|
||||
|
||||
## 📄 License
|
||||
|
||||
Same as main ThrillWiki project.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 8, 2025
|
||||
**Status**: Foundation Phase - Active Development
|
||||
0
django/config/__init__.py
Normal file
0
django/config/__init__.py
Normal file
16
django/config/asgi.py
Normal file
16
django/config/asgi.py
Normal file
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
ASGI config for config project.
|
||||
|
||||
It exposes the ASGI callable as a module-level variable named ``application``.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/4.2/howto/deployment/asgi/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from django.core.asgi import get_asgi_application
|
||||
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
|
||||
|
||||
application = get_asgi_application()
|
||||
123
django/config/settings.py
Normal file
123
django/config/settings.py
Normal file
@@ -0,0 +1,123 @@
|
||||
"""
|
||||
Django settings for config project.
|
||||
|
||||
Generated by 'django-admin startproject' using Django 4.2.8.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/4.2/topics/settings/
|
||||
|
||||
For the full list of settings and their values, see
|
||||
https://docs.djangoproject.com/en/4.2/ref/settings/
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
# Build paths inside the project like this: BASE_DIR / 'subdir'.
|
||||
BASE_DIR = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
# Quick-start development settings - unsuitable for production
|
||||
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/
|
||||
|
||||
# SECURITY WARNING: keep the secret key used in production secret!
|
||||
SECRET_KEY = "django-insecure-6h8r1j#p%g5^x%7970n6fe&9)5o9e4p-i#_okjib7=2--#a8b="
|
||||
|
||||
# SECURITY WARNING: don't run with debug turned on in production!
|
||||
DEBUG = True
|
||||
|
||||
ALLOWED_HOSTS = []
|
||||
|
||||
|
||||
# Application definition
|
||||
|
||||
INSTALLED_APPS = [
|
||||
"django.contrib.admin",
|
||||
"django.contrib.auth",
|
||||
"django.contrib.contenttypes",
|
||||
"django.contrib.sessions",
|
||||
"django.contrib.messages",
|
||||
"django.contrib.staticfiles",
|
||||
]
|
||||
|
||||
MIDDLEWARE = [
|
||||
"django.middleware.security.SecurityMiddleware",
|
||||
"django.contrib.sessions.middleware.SessionMiddleware",
|
||||
"django.middleware.common.CommonMiddleware",
|
||||
"django.middleware.csrf.CsrfViewMiddleware",
|
||||
"django.contrib.auth.middleware.AuthenticationMiddleware",
|
||||
"django.contrib.messages.middleware.MessageMiddleware",
|
||||
"django.middleware.clickjacking.XFrameOptionsMiddleware",
|
||||
]
|
||||
|
||||
ROOT_URLCONF = "config.urls"
|
||||
|
||||
TEMPLATES = [
|
||||
{
|
||||
"BACKEND": "django.template.backends.django.DjangoTemplates",
|
||||
"DIRS": [],
|
||||
"APP_DIRS": True,
|
||||
"OPTIONS": {
|
||||
"context_processors": [
|
||||
"django.template.context_processors.debug",
|
||||
"django.template.context_processors.request",
|
||||
"django.contrib.auth.context_processors.auth",
|
||||
"django.contrib.messages.context_processors.messages",
|
||||
],
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
WSGI_APPLICATION = "config.wsgi.application"
|
||||
|
||||
|
||||
# Database
|
||||
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases
|
||||
|
||||
DATABASES = {
|
||||
"default": {
|
||||
"ENGINE": "django.db.backends.sqlite3",
|
||||
"NAME": BASE_DIR / "db.sqlite3",
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
# Password validation
|
||||
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators
|
||||
|
||||
AUTH_PASSWORD_VALIDATORS = [
|
||||
{
|
||||
"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
|
||||
},
|
||||
{
|
||||
"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
|
||||
},
|
||||
{
|
||||
"NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
|
||||
},
|
||||
{
|
||||
"NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
# Internationalization
|
||||
# https://docs.djangoproject.com/en/4.2/topics/i18n/
|
||||
|
||||
LANGUAGE_CODE = "en-us"
|
||||
|
||||
TIME_ZONE = "UTC"
|
||||
|
||||
USE_I18N = True
|
||||
|
||||
USE_TZ = True
|
||||
|
||||
|
||||
# Static files (CSS, JavaScript, Images)
|
||||
# https://docs.djangoproject.com/en/4.2/howto/static-files/
|
||||
|
||||
STATIC_URL = "static/"
|
||||
|
||||
# Default primary key field type
|
||||
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field
|
||||
|
||||
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
|
||||
23
django/config/urls.py
Normal file
23
django/config/urls.py
Normal file
@@ -0,0 +1,23 @@
|
||||
"""
|
||||
URL configuration for config project.
|
||||
|
||||
The `urlpatterns` list routes URLs to views. For more information please see:
|
||||
https://docs.djangoproject.com/en/4.2/topics/http/urls/
|
||||
Examples:
|
||||
Function views
|
||||
1. Add an import: from my_app import views
|
||||
2. Add a URL to urlpatterns: path('', views.home, name='home')
|
||||
Class-based views
|
||||
1. Add an import: from other_app.views import Home
|
||||
2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
|
||||
Including another URLconf
|
||||
1. Import the include() function: from django.urls import include, path
|
||||
2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
|
||||
"""
|
||||
|
||||
from django.contrib import admin
|
||||
from django.urls import path
|
||||
|
||||
urlpatterns = [
|
||||
path("admin/", admin.site.urls),
|
||||
]
|
||||
16
django/config/wsgi.py
Normal file
16
django/config/wsgi.py
Normal file
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
WSGI config for config project.
|
||||
|
||||
It exposes the WSGI callable as a module-level variable named ``application``.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/4.2/howto/deployment/wsgi/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from django.core.wsgi import get_wsgi_application
|
||||
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
|
||||
|
||||
application = get_wsgi_application()
|
||||
22
django/manage.py
Executable file
22
django/manage.py
Executable file
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env python
|
||||
"""Django's command-line utility for administrative tasks."""
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def main():
|
||||
"""Run administrative tasks."""
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
|
||||
try:
|
||||
from django.core.management import execute_from_command_line
|
||||
except ImportError as exc:
|
||||
raise ImportError(
|
||||
"Couldn't import Django. Are you sure it's installed and "
|
||||
"available on your PYTHONPATH environment variable? Did you "
|
||||
"forget to activate a virtual environment?"
|
||||
) from exc
|
||||
execute_from_command_line(sys.argv)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
64
django/requirements/base.txt
Normal file
64
django/requirements/base.txt
Normal file
@@ -0,0 +1,64 @@
|
||||
# Core Django
|
||||
Django==4.2.8
|
||||
psycopg[binary]==3.2.3
|
||||
|
||||
# API Framework (django-ninja for FastAPI-style performance)
|
||||
django-ninja==1.1.0
|
||||
pydantic==2.10.6
|
||||
|
||||
# Database & ORM utilities
|
||||
django-model-utils==4.3.1
|
||||
django-guardian==2.4.0
|
||||
django-lifecycle==1.0.0
|
||||
django-fsm==2.8.1
|
||||
django-dirtyfields==1.9.2
|
||||
|
||||
# Async & Background Tasks
|
||||
celery[redis]==5.3.4
|
||||
django-celery-beat==2.5.0
|
||||
django-celery-results==2.5.1
|
||||
flower==2.0.1
|
||||
|
||||
# Caching & Performance
|
||||
django-redis==5.4.0
|
||||
django-cacheops==7.0.2
|
||||
hiredis==2.3.2
|
||||
|
||||
# Real-time
|
||||
channels==4.0.0
|
||||
channels-redis==4.1.0
|
||||
daphne==4.0.0
|
||||
|
||||
# Media & Storage
|
||||
django-storages[s3]==1.14.2
|
||||
Pillow==11.0.0
|
||||
python-magic==0.4.27
|
||||
|
||||
# Security
|
||||
django-cors-headers==4.3.1
|
||||
django-ratelimit==4.1.0
|
||||
django-otp==1.3.0
|
||||
django-allauth==0.58.2
|
||||
djangorestframework-simplejwt==5.3.1
|
||||
django-defender==0.9.7
|
||||
|
||||
# Validation & Serialization
|
||||
marshmallow==3.20.1
|
||||
|
||||
# Utilities
|
||||
django-extensions==3.2.3
|
||||
django-environ==0.11.2
|
||||
django-filter==23.5
|
||||
python-slugify==8.0.1
|
||||
python-dateutil==2.8.2
|
||||
|
||||
# Monitoring & Logging
|
||||
sentry-sdk==1.39.1
|
||||
structlog==23.2.0
|
||||
|
||||
# HTTP & External APIs
|
||||
requests==2.31.0
|
||||
httpx==0.25.2
|
||||
|
||||
# UUID utilities
|
||||
shortuuid==1.0.11
|
||||
23
django/requirements/local.txt
Normal file
23
django/requirements/local.txt
Normal file
@@ -0,0 +1,23 @@
|
||||
-r base.txt
|
||||
|
||||
# Development & Debugging
|
||||
django-debug-toolbar==4.2.0
|
||||
django-silk==5.0.4
|
||||
ipython==8.18.1
|
||||
ipdb==0.13.13
|
||||
|
||||
# Testing
|
||||
pytest==7.4.3
|
||||
pytest-django==4.7.0
|
||||
pytest-cov==4.1.0
|
||||
pytest-xdist==3.5.0
|
||||
factory-boy==3.3.0
|
||||
Faker==20.1.0
|
||||
coverage==7.3.3
|
||||
|
||||
# Code Quality
|
||||
black==23.12.1
|
||||
flake8==6.1.0
|
||||
isort==5.13.2
|
||||
mypy==1.7.1
|
||||
django-stubs==4.2.7
|
||||
11
django/requirements/production.txt
Normal file
11
django/requirements/production.txt
Normal file
@@ -0,0 +1,11 @@
|
||||
-r base.txt
|
||||
|
||||
# Production Web Server
|
||||
gunicorn==21.2.0
|
||||
whitenoise==6.6.0
|
||||
|
||||
# Production monitoring
|
||||
django-prometheus==2.3.1
|
||||
|
||||
# Security
|
||||
cryptography==41.0.7
|
||||
Reference in New Issue
Block a user