Add secret management guide, client-side performance monitoring, and search accessibility enhancements

- Introduced a comprehensive Secret Management Guide detailing best practices, secret classification, development setup, production management, rotation procedures, and emergency protocols.
- Implemented a client-side performance monitoring script to track various metrics including page load performance, paint metrics, layout shifts, and memory usage.
- Enhanced search accessibility with keyboard navigation support for search results, ensuring compliance with WCAG standards and improving user experience.
This commit is contained in:
pacnpal
2025-12-23 16:41:42 -05:00
parent ae31e889d7
commit edcd8f2076
155 changed files with 22046 additions and 4645 deletions

View File

@@ -0,0 +1,259 @@
# ThrillWiki Configuration System
This document provides an overview of the ThrillWiki configuration system, including how settings are organized, how to configure different environments, and best practices for managing configuration.
## Architecture Overview
ThrillWiki uses a modular configuration architecture that separates concerns and makes settings easy to manage across different environments.
### Directory Structure
```
backend/config/
├── django/ # Environment-specific settings
│ ├── base.py # Core Django settings (imports from settings/)
│ ├── local.py # Local development settings
│ ├── production.py # Production settings
│ └── test.py # Test settings
├── settings/ # Modular settings modules
│ ├── __init__.py # Package documentation
│ ├── cache.py # Redis caching configuration
│ ├── database.py # Database and GeoDjango settings
│ ├── email.py # Email backend configuration
│ ├── logging.py # Logging formatters, handlers, loggers
│ ├── rest_framework.py # DRF, JWT, CORS, API docs
│ ├── secrets.py # Secret management utilities
│ ├── security.py # Security headers and authentication
│ ├── storage.py # Static/media files and WhiteNoise
│ ├── third_party.py # Allauth, Celery, Cloudflare, etc.
│ └── validation.py # Environment variable validation
└── celery.py # Celery task queue configuration
```
### How Settings Are Loaded
1. **base.py** defines core Django settings and imports all modular settings
2. Environment-specific files (**local.py**, **production.py**, **test.py**) extend base.py
3. The active settings module is determined by `DJANGO_SETTINGS_MODULE`
4. Modular settings use **python-decouple** to read environment variables
### Environment Selection
The settings module is selected in this order:
1. `DJANGO_SETTINGS_MODULE` environment variable (explicit override)
2. Test command detection (`manage.py test``config.django.test`)
3. Production indicators (cloud provider environment variables)
4. `DEBUG=False``config.django.production`
5. Default → `config.django.local`
## Configuration Methods
### Using Environment Variables
Environment variables are the primary configuration method. Create a `.env` file from the template:
```bash
cp .env.example .env
# Edit .env with your values
```
### python-decouple
All settings modules use python-decouple for consistent environment variable handling:
```python
from decouple import config
# String with default
DEBUG = config("DEBUG", default=True, cast=bool)
# Integer with validation
PORT = config("PORT", default=8000, cast=int)
# List from comma-separated string
ALLOWED_HOSTS = config(
"ALLOWED_HOSTS",
default="localhost",
cast=lambda v: [s.strip() for s in v.split(",")]
)
```
## Environment-Specific Configuration
### Local Development (`config.django.local`)
- Debug mode enabled
- Local memory cache (no Redis required)
- Console email backend
- Development middleware (nplusone, debug toolbar)
- Relaxed security settings
### Production (`config.django.production`)
- Debug mode disabled
- Redis caching required
- Strict security settings (HSTS, secure cookies)
- JSON logging for log aggregation
- Sentry integration (optional)
### Test (`config.django.test`)
- In-memory SpatiaLite database
- In-memory cache
- Fast password hashing
- Logging disabled
- Celery tasks run synchronously
## Modular Settings Reference
### database.py
- `DATABASES` - Database connection configuration
- `GDAL_LIBRARY_PATH`, `GEOS_LIBRARY_PATH` - GeoDjango libraries
- `CONN_MAX_AGE` - Connection pooling
- `DATABASE_OPTIONS` - PostgreSQL-specific settings
### cache.py
- `CACHES` - Redis cache backends (default, sessions, api)
- `SESSION_*` - Session storage settings
- `CACHE_MIDDLEWARE_*` - Cache middleware settings
### security.py
- `SECURE_*` - Security header settings
- `SESSION_COOKIE_*`, `CSRF_COOKIE_*` - Cookie security
- `AUTH_PASSWORD_VALIDATORS` - Password validation rules
- `PERMISSIONS_POLICY` - Browser feature permissions
### email.py
- `EMAIL_BACKEND` - Email sending backend
- `FORWARD_EMAIL_*` - ForwardEmail configuration
- SMTP settings for custom email servers
### rest_framework.py
- `REST_FRAMEWORK` - DRF configuration
- `CORS_*` - Cross-origin settings
- `SIMPLE_JWT` - JWT token settings
- `REST_AUTH` - dj-rest-auth settings
- `SPECTACULAR_SETTINGS` - OpenAPI documentation
### third_party.py
- `ACCOUNT_*`, `SOCIALACCOUNT_*` - Allauth settings
- `CLOUDFLARE_IMAGES` - Image CDN configuration
- `ROADTRIP_*` - Road trip service settings
- `HEALTH_CHECK` - Health check thresholds
### storage.py
- `STATIC_*`, `MEDIA_*` - File serving configuration
- `STORAGES` - Django 4.2+ storage backends
- `WHITENOISE_*` - Static file optimization
- `FILE_UPLOAD_*` - Upload security limits
### logging.py
- `LOGGING` - Complete logging configuration
- Formatters: verbose, json, simple
- Handlers: console, file, error_file, performance
- Loggers for Django, application, and third-party
## Secret Management
### Validation
Use the management command to validate configuration:
```bash
# Full validation
python manage.py validate_settings
# Strict mode (warnings become errors)
python manage.py validate_settings --strict
# JSON output
python manage.py validate_settings --json
# Secrets only
python manage.py validate_settings --secrets-only
```
### Required Secrets
These secrets must be set in all environments:
| Secret | Description |
|--------|-------------|
| `SECRET_KEY` | Django cryptographic signing key (50+ chars) |
| `DATABASE_URL` | Database connection URL |
### Secret Rotation
For production environments:
1. Enable rotation checking: `SECRET_ROTATION_ENABLED=True`
2. Track versions: `SECRET_KEY_VERSION=1`
3. Monitor expiry warnings in logs
4. Rotate secrets before expiry
See [Secret Management Guide](./secret-management.md) for detailed procedures.
## Troubleshooting
### Common Issues
**Settings module not found:**
```
ModuleNotFoundError: No module named 'config.django.local'
```
Ensure you're running commands from the `backend/` directory.
**Required environment variable missing:**
```
decouple.UndefinedValueError: SECRET_KEY not found
```
Create a `.env` file or set the environment variable.
**Database connection failed:**
```
django.db.utils.OperationalError: could not connect to server
```
Check `DATABASE_URL` format and database server status.
### Validation Errors
Run validation to identify configuration issues:
```bash
python manage.py validate_settings
```
### Debug Configuration
To debug configuration loading:
```python
# In Django shell
from django.conf import settings
print(settings.DEBUG)
print(settings.DATABASES)
```
## Best Practices
1. **Never commit secrets** - Use `.env` files or secret management services
2. **Use environment variables** - Don't hardcode configuration values
3. **Validate on startup** - Catch configuration errors early
4. **Separate environments** - Use different settings for dev/staging/production
5. **Document custom settings** - Add comments explaining non-obvious configuration
6. **Use appropriate defaults** - Secure defaults for production, convenient for development
## Related Documentation
- [Environment Variables Reference](./environment-variables.md)
- [Secret Management Guide](./secret-management.md)
- [Deployment Guide](./deployment.md)

View File

@@ -0,0 +1,501 @@
# Deployment Guide
This guide covers deploying ThrillWiki to production environments, including configuration, security, and monitoring setup.
## Pre-Deployment Checklist
### Configuration
- [ ] All required environment variables set
- [ ] `DEBUG=False`
- [ ] Strong `SECRET_KEY` (50+ characters, randomly generated)
- [ ] `ALLOWED_HOSTS` properly configured
- [ ] `CSRF_TRUSTED_ORIGINS` set
- [ ] Database URL points to production database
- [ ] Redis URL configured for caching
### Security
- [ ] `SECURE_SSL_REDIRECT=True`
- [ ] `SESSION_COOKIE_SECURE=True`
- [ ] `CSRF_COOKIE_SECURE=True`
- [ ] `SECURE_HSTS_SECONDS` set (31536000 recommended)
- [ ] `SECURE_HSTS_INCLUDE_SUBDOMAINS=True`
- [ ] No debug tools enabled
- [ ] Turnstile keys configured
### Validation
Run the configuration validator:
```bash
python manage.py validate_settings --strict
python manage.py check --deploy
```
## Environment Configuration
### Required Environment Variables
```bash
# Core Django
SECRET_KEY=<generate-secure-key>
DEBUG=False
DJANGO_SETTINGS_MODULE=config.django.production
ALLOWED_HOSTS=thrillwiki.com,www.thrillwiki.com
CSRF_TRUSTED_ORIGINS=https://thrillwiki.com,https://www.thrillwiki.com
# Database
DATABASE_URL=postgis://user:pass@db-host:5432/thrillwiki
# Cache
REDIS_URL=redis://:password@redis-host:6379/1
# Email
EMAIL_BACKEND=django_forwardemail.backends.ForwardEmailBackend
FORWARD_EMAIL_API_KEY=your-api-key
FORWARD_EMAIL_DOMAIN=thrillwiki.com
# Security
TURNSTILE_SITE_KEY=your-site-key
TURNSTILE_SECRET_KEY=your-secret-key
SECURE_SSL_REDIRECT=True
SESSION_COOKIE_SECURE=True
CSRF_COOKIE_SECURE=True
# Cloudflare Images
CLOUDFLARE_IMAGES_ACCOUNT_ID=your-account-id
CLOUDFLARE_IMAGES_API_TOKEN=your-api-token
CLOUDFLARE_IMAGES_ACCOUNT_HASH=your-account-hash
# Road Trip Service
ROADTRIP_USER_AGENT=ThrillWiki/1.0 (https://thrillwiki.com)
# Frontend
FRONTEND_DOMAIN=https://thrillwiki.com
```
### Optional Production Settings
```bash
# Monitoring
SENTRY_DSN=https://xxx@xxx.ingest.sentry.io/xxx
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
# Secret rotation
SECRET_ROTATION_ENABLED=True
SECRET_KEY_VERSION=1
# Performance tuning
DATABASE_CONN_MAX_AGE=600
REDIS_MAX_CONNECTIONS=100
```
## Deployment Platforms
### Docker Deployment
Create a production Dockerfile:
```dockerfile
FROM python:3.11-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
libpq-dev \
libgdal-dev \
libgeos-dev \
&& rm -rf /var/lib/apt/lists/*
# Install Python dependencies
COPY backend/pyproject.toml backend/uv.lock ./
RUN pip install uv && uv sync --frozen
# Copy application
COPY backend/ .
# Collect static files
RUN python manage.py collectstatic --noinput
# Run with Gunicorn
CMD ["gunicorn", "--bind", "0.0.0.0:8000", "--workers", "4", "thrillwiki.wsgi:application"]
```
Docker Compose for production:
```yaml
version: '3.8'
services:
web:
build: .
environment:
- DJANGO_SETTINGS_MODULE=config.django.production
env_file:
- .env
ports:
- "8000:8000"
depends_on:
- db
- redis
db:
image: postgis/postgis:14-3.3
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=thrillwiki
- POSTGRES_USER=thrillwiki
- POSTGRES_PASSWORD=${DB_PASSWORD}
redis:
image: redis:7-alpine
command: redis-server --requirepass ${REDIS_PASSWORD}
volumes:
- redis_data:/data
celery:
build: .
command: celery -A config.celery worker -l info
env_file:
- .env
depends_on:
- db
- redis
celery-beat:
build: .
command: celery -A config.celery beat -l info
env_file:
- .env
depends_on:
- db
- redis
volumes:
postgres_data:
redis_data:
```
### Heroku Deployment
```bash
# Create app
heroku create thrillwiki
# Add buildpacks
heroku buildpacks:add heroku/python
heroku buildpacks:add https://github.com/heroku/heroku-geo-buildpack.git
# Add add-ons
heroku addons:create heroku-postgresql:standard-0
heroku addons:create heroku-redis:premium-0
# Set environment variables
heroku config:set DJANGO_SETTINGS_MODULE=config.django.production
heroku config:set SECRET_KEY=$(python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())")
heroku config:set DEBUG=False
# ... set other variables
# Deploy
git push heroku main
# Run migrations
heroku run python manage.py migrate
```
### AWS ECS Deployment
Task definition example:
```json
{
"family": "thrillwiki",
"containerDefinitions": [
{
"name": "web",
"image": "your-ecr-repo/thrillwiki:latest",
"portMappings": [
{
"containerPort": 8000,
"protocol": "tcp"
}
],
"environment": [
{
"name": "DJANGO_SETTINGS_MODULE",
"value": "config.django.production"
}
],
"secrets": [
{
"name": "SECRET_KEY",
"valueFrom": "arn:aws:secretsmanager:region:account:secret:thrillwiki/SECRET_KEY"
},
{
"name": "DATABASE_URL",
"valueFrom": "arn:aws:secretsmanager:region:account:secret:thrillwiki/DATABASE_URL"
}
]
}
]
}
```
## Database Setup
### PostgreSQL with PostGIS
```sql
-- Create database
CREATE DATABASE thrillwiki;
-- Enable PostGIS
\c thrillwiki
CREATE EXTENSION postgis;
CREATE EXTENSION postgis_topology;
-- Create user
CREATE USER thrillwiki_app WITH ENCRYPTED PASSWORD 'secure-password';
GRANT ALL PRIVILEGES ON DATABASE thrillwiki TO thrillwiki_app;
```
### Run Migrations
```bash
python manage.py migrate
python manage.py createsuperuser
```
## Static Files
### Collect Static Files
```bash
python manage.py collectstatic --noinput
```
### WhiteNoise Configuration
WhiteNoise is configured for production static file serving. No additional web server configuration needed for static files.
### CDN Integration (Optional)
For CDN, update `STATIC_URL`:
```bash
STATIC_URL=https://cdn.thrillwiki.com/static/
```
## Monitoring Setup
### Sentry Integration
```bash
SENTRY_DSN=https://xxx@xxx.ingest.sentry.io/xxx
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
```
### Health Checks
Health check endpoint: `/health/`
```bash
# Test health check
curl https://thrillwiki.com/health/
```
Configure load balancer health checks to use this endpoint.
### Logging
Production logging outputs JSON to stdout (suitable for log aggregation):
```json
{
"levelname": "INFO",
"asctime": "2024-01-15 10:30:00,000",
"module": "views",
"message": "Request processed"
}
```
## Performance Tuning
### Database Connection Pooling
```bash
DATABASE_CONN_MAX_AGE=600 # 10 minutes
DATABASE_CONNECT_TIMEOUT=10
DATABASE_STATEMENT_TIMEOUT=30000
```
### Redis Connection Pooling
```bash
REDIS_MAX_CONNECTIONS=100
REDIS_CONNECTION_TIMEOUT=20
```
### Gunicorn Configuration
```bash
# gunicorn.conf.py
workers = 4 # (2 * CPU cores) + 1
worker_class = "gevent"
worker_connections = 1000
timeout = 30
keepalive = 5
max_requests = 1000
max_requests_jitter = 50
```
## Celery Workers
### Start Workers
```bash
# Main worker
celery -A config.celery worker -l info
# Beat scheduler
celery -A config.celery beat -l info
# With specific queues
celery -A config.celery worker -l info -Q default,trending,analytics,cache
```
### Supervisor Configuration
```ini
[program:celery]
command=/app/.venv/bin/celery -A config.celery worker -l info
directory=/app
user=www-data
numprocs=1
autostart=true
autorestart=true
stdout_logfile=/var/log/celery/worker.log
stderr_logfile=/var/log/celery/worker-error.log
[program:celery-beat]
command=/app/.venv/bin/celery -A config.celery beat -l info
directory=/app
user=www-data
numprocs=1
autostart=true
autorestart=true
stdout_logfile=/var/log/celery/beat.log
stderr_logfile=/var/log/celery/beat-error.log
```
## SSL/TLS Configuration
### Nginx Configuration
```nginx
server {
listen 443 ssl http2;
server_name thrillwiki.com www.thrillwiki.com;
ssl_certificate /etc/letsencrypt/live/thrillwiki.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/thrillwiki.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
server {
listen 80;
server_name thrillwiki.com www.thrillwiki.com;
return 301 https://$server_name$request_uri;
}
```
## Post-Deployment Verification
### Functional Tests
```bash
# API health
curl -s https://thrillwiki.com/api/health/ | jq
# Static files
curl -I https://thrillwiki.com/static/css/tailwind.css
# API endpoint
curl -s https://thrillwiki.com/api/v1/parks/ | jq
```
### Security Headers
```bash
# Check security headers
curl -I https://thrillwiki.com/ | grep -E "(Strict-Transport|X-Frame|X-Content|Content-Security)"
```
### Performance
```bash
# Response time
time curl -s https://thrillwiki.com/api/v1/parks/ > /dev/null
```
## Rollback Procedure
If deployment fails:
1. **Immediate rollback:**
```bash
# Docker
docker-compose down
docker-compose -f docker-compose.previous.yml up -d
# Heroku
heroku rollback
# Kubernetes
kubectl rollout undo deployment/thrillwiki
```
2. **Database rollback (if needed):**
```bash
python manage.py migrate app_name migration_number
```
3. **Investigate and fix:**
- Check error logs
- Review configuration differences
- Test in staging environment
## Maintenance
### Regular Tasks
- **Daily:** Review error logs, check health endpoints
- **Weekly:** Review performance metrics, check disk usage
- **Monthly:** Rotate secrets, review security updates
- **Quarterly:** Full security audit, dependency updates
### Backup Procedures
```bash
# Database backup
pg_dump thrillwiki > backup_$(date +%Y%m%d).sql
# Media files backup
aws s3 sync /app/media s3://thrillwiki-backups/media/
```

View File

@@ -0,0 +1,338 @@
# Environment Variables Reference
Complete reference for all environment variables used in ThrillWiki.
## Quick Start
1. Copy the example file: `cp .env.example .env`
2. Edit `.env` with your values
3. Run validation: `python manage.py validate_settings`
## Required Variables
These must be set in all environments.
| Variable | Type | Description |
|----------|------|-------------|
| `SECRET_KEY` | string | Django secret key (minimum 50 characters) |
| `DATABASE_URL` | url | Database connection URL |
## Production-Required Variables
These variables **must** be explicitly set for production deployments. The application will not function correctly in production without them.
> **Important:** See [`.env.example`](../../.env.example) for example values and [`PRODUCTION_CHECKLIST.md`](../PRODUCTION_CHECKLIST.md) for deployment verification steps.
| Variable | Type | Format/Example | Description |
|----------|------|----------------|-------------|
| `DEBUG` | bool | `False` | **Must be `False` in production.** Enables detailed error pages and debug toolbar when `True`. |
| `DJANGO_SETTINGS_MODULE` | string | `config.django.production` | **Must use `config.django.production` for production.** Controls which settings module Django loads. |
| `ALLOWED_HOSTS` | list | `thrillwiki.com,www.thrillwiki.com` | **Required.** Comma-separated list of valid hostnames. No default in production - requests will fail without this. |
| `CSRF_TRUSTED_ORIGINS` | list | `https://thrillwiki.com,https://www.thrillwiki.com` | **Required.** Must include `https://` prefix. CSRF validation fails without proper configuration. |
| `REDIS_URL` | url | `redis://[:password@]host:6379/0` | **Required for caching and sessions.** Production uses Redis for caching, session storage, and Celery broker. |
### Production Settings Behavior
When `DJANGO_SETTINGS_MODULE=config.django.production`:
- `DEBUG` is always `False`
- `ALLOWED_HOSTS` has **no default** - must be explicitly set
- `CSRF_TRUSTED_ORIGINS` has **no default** - must be explicitly set
- SSL redirect is enforced (`SECURE_SSL_REDIRECT=True`)
- Secure cookies are enforced (`SESSION_COOKIE_SECURE=True`, `CSRF_COOKIE_SECURE=True`)
- HSTS is enabled with 1-year max-age
- Redis cache is required (application will fail if `REDIS_URL` is not set)
### Validation
Run the following to validate your production configuration:
```bash
DJANGO_SETTINGS_MODULE=config.django.production python manage.py check --deploy
```
## Core Django Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `DEBUG` | bool | `True` | Enable debug mode |
| `DJANGO_SETTINGS_MODULE` | string | `config.django.local` | Settings module to use |
| `ALLOWED_HOSTS` | list | `localhost,127.0.0.1` | Comma-separated allowed hosts |
| `CSRF_TRUSTED_ORIGINS` | list | `` | Comma-separated trusted origins |
## Database Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `DATABASE_URL` | url | (required) | Database connection URL |
| `DATABASE_CONN_MAX_AGE` | int | `600` | Connection pool timeout (seconds) |
| `DATABASE_CONNECT_TIMEOUT` | int | `10` | Connection timeout (seconds) |
| `DATABASE_STATEMENT_TIMEOUT` | int | `30000` | Query timeout (milliseconds) |
| `DATABASE_READ_REPLICA_URL` | url | `` | Optional read replica URL |
### Database URL Formats
```bash
# PostgreSQL with PostGIS
DATABASE_URL=postgis://user:password@host:5432/dbname
# PostgreSQL without PostGIS
DATABASE_URL=postgres://user:password@host:5432/dbname
# SQLite (development only)
DATABASE_URL=sqlite:///path/to/db.sqlite3
```
## GeoDjango Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `GDAL_LIBRARY_PATH` | path | `/opt/homebrew/lib/libgdal.dylib` | Path to GDAL library |
| `GEOS_LIBRARY_PATH` | path | `/opt/homebrew/lib/libgeos_c.dylib` | Path to GEOS library |
### Platform-Specific Paths
```bash
# macOS (Homebrew)
GDAL_LIBRARY_PATH=/opt/homebrew/lib/libgdal.dylib
GEOS_LIBRARY_PATH=/opt/homebrew/lib/libgeos_c.dylib
# Linux (Ubuntu/Debian)
GDAL_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/libgdal.so
GEOS_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/libgeos_c.so
```
## Cache Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `REDIS_URL` | url | `redis://127.0.0.1:6379/1` | Redis connection URL |
| `REDIS_SESSIONS_URL` | url | (from REDIS_URL) | Redis URL for sessions |
| `REDIS_API_URL` | url | (from REDIS_URL) | Redis URL for API cache |
| `REDIS_MAX_CONNECTIONS` | int | `100` | Maximum Redis connections |
| `REDIS_CONNECTION_TIMEOUT` | int | `20` | Redis connection timeout |
| `REDIS_IGNORE_EXCEPTIONS` | bool | `True` | Graceful degradation |
| `CACHE_MIDDLEWARE_SECONDS` | int | `300` | Cache middleware timeout |
| `CACHE_MIDDLEWARE_KEY_PREFIX` | string | `thrillwiki` | Cache key prefix |
| `CACHE_KEY_PREFIX` | string | `thrillwiki` | Default cache key prefix |
## Email Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `EMAIL_BACKEND` | string | `django_forwardemail.backends.ForwardEmailBackend` | Email backend class |
| `SERVER_EMAIL` | email | `django_webmaster@thrillwiki.com` | Server email address |
| `DEFAULT_FROM_EMAIL` | string | `ThrillWiki <noreply@thrillwiki.com>` | Default from address |
| `EMAIL_SUBJECT_PREFIX` | string | `[ThrillWiki]` | Subject prefix |
| `EMAIL_TIMEOUT` | int | `30` | Email operation timeout |
### ForwardEmail Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `FORWARD_EMAIL_BASE_URL` | url | `https://api.forwardemail.net` | API base URL |
| `FORWARD_EMAIL_API_KEY` | string | `` | API key |
| `FORWARD_EMAIL_DOMAIN` | string | `` | Sending domain |
### SMTP Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `EMAIL_HOST` | string | `localhost` | SMTP server |
| `EMAIL_PORT` | int | `587` | SMTP port |
| `EMAIL_USE_TLS` | bool | `True` | Use TLS |
| `EMAIL_USE_SSL` | bool | `False` | Use SSL |
| `EMAIL_HOST_USER` | string | `` | SMTP username |
| `EMAIL_HOST_PASSWORD` | string | `` | SMTP password |
## Security Settings
### SSL/HTTPS
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SECURE_SSL_REDIRECT` | bool | `False` | Redirect HTTP to HTTPS |
| `SECURE_PROXY_SSL_HEADER` | tuple | `` | Proxy SSL header |
### HSTS (HTTP Strict Transport Security)
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SECURE_HSTS_SECONDS` | int | `31536000` | HSTS max-age |
| `SECURE_HSTS_INCLUDE_SUBDOMAINS` | bool | `True` | Include subdomains |
| `SECURE_HSTS_PRELOAD` | bool | `False` | Allow preload |
| `SECURE_REDIRECT_EXEMPT` | list | `` | URLs exempt from redirect |
### Security Headers
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SECURE_BROWSER_XSS_FILTER` | bool | `True` | XSS filter header |
| `SECURE_CONTENT_TYPE_NOSNIFF` | bool | `True` | Nosniff header |
| `X_FRAME_OPTIONS` | string | `DENY` | Frame options |
| `SECURE_REFERRER_POLICY` | string | `strict-origin-when-cross-origin` | Referrer policy |
| `SECURE_CROSS_ORIGIN_OPENER_POLICY` | string | `same-origin` | COOP header |
### Cookie Security
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SESSION_COOKIE_SECURE` | bool | `False` | HTTPS-only session cookie |
| `SESSION_COOKIE_HTTPONLY` | bool | `True` | No JS access |
| `SESSION_COOKIE_SAMESITE` | string | `Lax` | SameSite attribute |
| `SESSION_COOKIE_AGE` | int | `3600` | Cookie age (seconds) |
| `CSRF_COOKIE_SECURE` | bool | `False` | HTTPS-only CSRF cookie |
| `CSRF_COOKIE_HTTPONLY` | bool | `True` | No JS access |
| `CSRF_COOKIE_SAMESITE` | string | `Lax` | SameSite attribute |
### Cloudflare Turnstile
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `TURNSTILE_SITE_KEY` | string | `` | Turnstile site key |
| `TURNSTILE_SECRET_KEY` | string | `` | Turnstile secret key |
| `TURNSTILE_VERIFY_URL` | url | `https://challenges.cloudflare.com/turnstile/v0/siteverify` | Verification URL |
## API Configuration
### CORS
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `CORS_ALLOWED_ORIGINS` | list | `` | Allowed origins |
| `CORS_ALLOW_ALL_ORIGINS` | bool | `False` | Allow all origins |
### Rate Limiting
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `API_RATE_LIMIT_PER_MINUTE` | int | `60` | Requests per minute |
| `API_RATE_LIMIT_PER_HOUR` | int | `1000` | Requests per hour |
| `API_RATE_LIMIT_ANON_PER_MINUTE` | int | `60` | Anonymous rate limit |
| `API_RATE_LIMIT_USER_PER_HOUR` | int | `1000` | Authenticated rate limit |
### Pagination
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `API_PAGE_SIZE` | int | `20` | Default page size |
| `API_MAX_PAGE_SIZE` | int | `100` | Maximum page size |
| `API_VERSION` | string | `1.0.0` | API version string |
## JWT Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `JWT_ACCESS_TOKEN_LIFETIME_MINUTES` | int | `15` | Access token lifetime |
| `JWT_REFRESH_TOKEN_LIFETIME_DAYS` | int | `7` | Refresh token lifetime |
| `JWT_ISSUER` | string | `thrillwiki` | Token issuer |
## Cloudflare Images
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `CLOUDFLARE_IMAGES_ACCOUNT_ID` | string | `` | Account ID |
| `CLOUDFLARE_IMAGES_API_TOKEN` | string | `` | API token |
| `CLOUDFLARE_IMAGES_ACCOUNT_HASH` | string | `` | Account hash |
| `CLOUDFLARE_IMAGES_WEBHOOK_SECRET` | string | `` | Webhook secret |
| `CLOUDFLARE_IMAGES_DEFAULT_VARIANT` | string | `public` | Default variant |
| `CLOUDFLARE_IMAGES_UPLOAD_TIMEOUT` | int | `300` | Upload timeout |
| `CLOUDFLARE_IMAGES_CLEANUP_HOURS` | int | `24` | Cleanup interval |
| `CLOUDFLARE_IMAGES_MAX_FILE_SIZE` | int | `10485760` | Max file size (bytes) |
| `CLOUDFLARE_IMAGES_REQUIRE_SIGNED_URLS` | bool | `False` | Require signed URLs |
## Road Trip Service
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `ROADTRIP_USER_AGENT` | string | `ThrillWiki/1.0` | OSM API user agent |
| `ROADTRIP_CACHE_TIMEOUT` | int | `86400` | Geocoding cache timeout |
| `ROADTRIP_ROUTE_CACHE_TIMEOUT` | int | `21600` | Route cache timeout |
| `ROADTRIP_MAX_REQUESTS_PER_SECOND` | int | `1` | Rate limit |
| `ROADTRIP_REQUEST_TIMEOUT` | int | `10` | Request timeout |
| `ROADTRIP_MAX_RETRIES` | int | `3` | Max retries |
| `ROADTRIP_BACKOFF_FACTOR` | int | `2` | Retry backoff |
## Logging Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `LOG_DIR` | path | `logs` | Log directory |
| `ROOT_LOG_LEVEL` | string | `INFO` | Root log level |
| `DJANGO_LOG_LEVEL` | string | `WARNING` | Django log level |
| `DB_LOG_LEVEL` | string | `WARNING` | Database log level |
| `APP_LOG_LEVEL` | string | `INFO` | Application log level |
| `PERFORMANCE_LOG_LEVEL` | string | `INFO` | Performance log level |
| `QUERY_LOG_LEVEL` | string | `WARNING` | Query log level |
| `NPLUSONE_LOG_LEVEL` | string | `WARNING` | N+1 detection level |
| `REQUEST_LOG_LEVEL` | string | `INFO` | Request log level |
| `CELERY_LOG_LEVEL` | string | `INFO` | Celery log level |
| `CONSOLE_LOG_LEVEL` | string | `INFO` | Console output level |
| `FILE_LOG_LEVEL` | string | `INFO` | File output level |
| `FILE_LOG_FORMATTER` | string | `json` | File log format |
## Monitoring
### Sentry
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SENTRY_DSN` | url | `` | Sentry DSN |
| `SENTRY_ENVIRONMENT` | string | `production` | Sentry environment |
| `SENTRY_TRACES_SAMPLE_RATE` | float | `0.1` | Trace sampling rate |
### Health Checks
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `HEALTH_CHECK_DISK_USAGE_MAX` | int | `90` | Max disk usage % |
| `HEALTH_CHECK_MEMORY_MIN` | int | `100` | Min available memory MB |
## Feature Flags
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `ENABLE_DEBUG_TOOLBAR` | bool | `True` | Enable debug toolbar |
| `ENABLE_SILK_PROFILER` | bool | `False` | Enable Silk profiler |
| `TEMPLATES_ENABLED` | bool | `True` | Enable Django templates |
| `AUTOCOMPLETE_BLOCK_UNAUTHENTICATED` | bool | `False` | Require auth for autocomplete |
## File Upload Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `FILE_UPLOAD_MAX_MEMORY_SIZE` | int | `2621440` | Max in-memory upload (bytes) |
| `DATA_UPLOAD_MAX_MEMORY_SIZE` | int | `10485760` | Max request size (bytes) |
| `DATA_UPLOAD_MAX_NUMBER_FIELDS` | int | `1000` | Max form fields |
## WhiteNoise Settings
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `WHITENOISE_COMPRESSION_QUALITY` | int | `90` | Compression quality |
| `WHITENOISE_MAX_AGE` | int | `31536000` | Cache max-age |
| `WHITENOISE_MANIFEST_STRICT` | bool | `False` | Strict manifest mode |
## Secret Management
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `SECRET_ROTATION_ENABLED` | bool | `False` | Enable rotation checks |
| `SECRET_KEY_VERSION` | string | `1` | Current secret version |
| `SECRET_EXPIRY_WARNING_DAYS` | int | `30` | Warning threshold |
| `PASSWORD_MIN_LENGTH` | int | `8` | Minimum password length |
## Celery Configuration
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `CELERY_TASK_ALWAYS_EAGER` | bool | `False` | Run tasks synchronously |
| `CELERY_TASK_EAGER_PROPAGATES` | bool | `False` | Propagate task errors |
## Third-Party Integrations
| Variable | Type | Default | Description |
|----------|------|---------|-------------|
| `FRONTEND_DOMAIN` | url | `https://thrillwiki.com` | Frontend URL |
| `LOGIN_REDIRECT_URL` | path | `/` | Post-login redirect |
| `ACCOUNT_LOGOUT_REDIRECT_URL` | path | `/` | Post-logout redirect |
| `ACCOUNT_EMAIL_VERIFICATION` | string | `mandatory` | Email verification mode |

View File

@@ -0,0 +1,330 @@
# Secret Management Guide
This guide covers best practices for managing secrets in ThrillWiki, including rotation procedures, integration with secret management services, and emergency procedures.
## Overview
Secrets are sensitive configuration values that must be protected:
- Database credentials
- API keys and tokens
- Encryption keys
- Service passwords
ThrillWiki provides a flexible secret management system that can work with environment variables (development) or integrate with enterprise secret management services (production).
## Secret Classification
### Critical Secrets (Immediate rotation required if compromised)
| Secret | Impact | Rotation Frequency |
|--------|--------|-------------------|
| `SECRET_KEY` | All cryptographic operations | 90 days recommended |
| `DATABASE_URL` | Database access | On suspected breach |
| `SENTRY_DSN` | Error tracking access | On suspected breach |
### High-Priority Secrets
| Secret | Impact | Rotation Frequency |
|--------|--------|-------------------|
| `CLOUDFLARE_IMAGES_API_TOKEN` | Image CDN access | 180 days |
| `FORWARD_EMAIL_API_KEY` | Email sending | 180 days |
| `TURNSTILE_SECRET_KEY` | CAPTCHA bypass | 365 days |
### Service Secrets
| Secret | Impact | Rotation Frequency |
|--------|--------|-------------------|
| `CLOUDFLARE_IMAGES_WEBHOOK_SECRET` | Webhook verification | 365 days |
| `REDIS_URL` (with password) | Cache access | On suspected breach |
## Development Setup
For development, use a `.env` file:
```bash
# Copy the template
cp .env.example .env
# Generate a secure SECRET_KEY
python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
# Edit .env with your values
```
### Validation
Validate your configuration:
```bash
python manage.py validate_settings
```
## Production Secret Management
### Option 1: Environment Variables (Simple)
Set secrets as environment variables in your deployment platform:
```bash
# Heroku
heroku config:set SECRET_KEY="your-secret-key"
# Docker
docker run -e SECRET_KEY="your-secret-key" ...
# Kubernetes (from secret)
kubectl create secret generic thrillwiki-secrets \
--from-literal=SECRET_KEY="your-secret-key"
```
### Option 2: AWS Secrets Manager
For AWS deployments, integrate with Secrets Manager:
```python
# In config/settings/secrets.py, implement:
from config.settings.secrets import SecretProvider
class AWSSecretsProvider(SecretProvider):
def __init__(self, secret_name: str):
import boto3
self.client = boto3.client('secretsmanager')
self.secret_name = secret_name
self._cache = {}
def get_secret(self, name: str) -> str:
if not self._cache:
response = self.client.get_secret_value(
SecretId=self.secret_name
)
import json
self._cache = json.loads(response['SecretString'])
return self._cache.get(name)
# Usage in settings
from config.settings.secrets import set_secret_provider
set_secret_provider(AWSSecretsProvider('thrillwiki/production'))
```
### Option 3: HashiCorp Vault
For Vault integration:
```python
class VaultSecretsProvider(SecretProvider):
def __init__(self, vault_addr: str, token: str, path: str):
import hvac
self.client = hvac.Client(url=vault_addr, token=token)
self.path = path
def get_secret(self, name: str) -> str:
response = self.client.secrets.kv.read_secret_version(
path=self.path
)
return response['data']['data'].get(name)
```
## Secret Rotation Procedures
### Rotating SECRET_KEY
**Impact:** All sessions will be invalidated. Users will need to log in again.
1. **Prepare the new key:**
```bash
python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
```
2. **Update deployment:**
```bash
# Update environment variable
export SECRET_KEY="new-secret-key-here"
export SECRET_KEY_VERSION="2" # Increment version
```
3. **Deploy during low-traffic period**
4. **Clear session cache:**
```bash
python manage.py clearsessions
```
5. **Monitor for issues:**
- Check error rates
- Monitor login success rates
- Watch for authentication errors
### Rotating Database Credentials
**Impact:** Application downtime if not done carefully.
1. **Create new database user:**
```sql
CREATE USER thrillwiki_new WITH PASSWORD 'new-password';
GRANT ALL PRIVILEGES ON DATABASE thrillwiki TO thrillwiki_new;
```
2. **Update application:**
```bash
export DATABASE_URL="postgis://thrillwiki_new:new-password@host:5432/thrillwiki"
```
3. **Deploy and verify**
4. **Remove old user:**
```sql
DROP USER thrillwiki_old;
```
### Rotating API Keys
For third-party API keys:
1. **Generate new key** in the service's dashboard
2. **Update environment variable**
3. **Deploy**
4. **Revoke old key** in the service's dashboard
## Automated Rotation
### Enable Rotation Monitoring
```bash
# In .env
SECRET_ROTATION_ENABLED=True
SECRET_KEY_VERSION=1
SECRET_EXPIRY_WARNING_DAYS=30
```
### Check Rotation Status
```bash
python manage.py validate_settings --secrets-only
```
### Automated Alerts
Configure logging to alert on expiry warnings:
```python
# In your monitoring setup, watch for:
# Logger: security
# Level: WARNING
# Message contains: "Secret expiry"
```
## Emergency Procedures
### Suspected Credential Compromise
1. **Immediate Actions:**
- Rotate the compromised credential immediately
- Check access logs for unauthorized use
- Review related systems for compromise
2. **Investigation:**
- Identify the source of the leak
- Review access patterns
- Document the incident
3. **Remediation:**
- Fix the vulnerability that caused the leak
- Update security procedures
- Conduct a post-mortem
### Complete Secret Rotation (Breach Response)
If a major breach is suspected, rotate all secrets:
```bash
# 1. Generate new SECRET_KEY
python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
# 2. Create new database credentials
# 3. Regenerate all API keys in respective dashboards
# 4. Update all environment variables
# 5. Deploy
# 6. Clear all caches
python manage.py clear_cache # If you have this command
redis-cli FLUSHALL # Or clear Redis directly
# 7. Invalidate all sessions
python manage.py clearsessions
# 8. Revoke all old credentials
```
## Security Best Practices
### Do's
- ✅ Use different secrets for each environment
- ✅ Rotate secrets regularly (90-180 days)
- ✅ Use secret management services in production
- ✅ Monitor for secret exposure (git-secrets, truffleHog)
- ✅ Log secret access for audit trails
- ✅ Use strong, randomly generated secrets
### Don'ts
- ❌ Never commit secrets to version control
- ❌ Never share secrets via chat or email
- ❌ Never use the same secret across environments
- ❌ Never use default or example secrets in production
- ❌ Never log secret values (even partially)
## Audit and Compliance
### Access Logging
Secret access is logged to the security logger:
```python
# Logged events:
# - Secret validation failures
# - Secret rotation warnings
# - Invalid secret format detections
```
### Regular Audits
Monthly security checklist:
- [ ] Review secret rotation schedule
- [ ] Check for exposed secrets in logs
- [ ] Verify secret strength requirements
- [ ] Audit secret access patterns
- [ ] Test secret rotation procedures
## Troubleshooting
### "Secret validation failed"
```bash
# Check the specific error
python manage.py validate_settings
# Common issues:
# - Secret too short
# - Placeholder value detected
# - Missing required secret
```
### "SECRET_KEY does not meet requirements"
```bash
# Generate a proper key
python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
```
### Secret provider errors
```bash
# Test secret provider connectivity
python -c "
from config.settings.secrets import get_secret_provider
provider = get_secret_provider()
print(provider.list_secrets())
"
```