# Django Performance Enhancement Implementation Plan
## Executive Summary
This document provides a comprehensive implementation plan for enhancing the ThrillWiki Django application across three priority areas: **API Standardization**, **Performance Enhancement**, and **Monitoring & Observability**. The plan leverages existing Django modules and follows Django styleguide best practices while building upon the current project's solid architectural foundation.
## Current Project Analysis
### Existing Strengths
- ✅ **Django REST Framework Integration**: Comprehensive DRF setup with Input/Output serializer patterns
- ✅ **Service Layer Architecture**: Well-implemented service/selector pattern following Django styleguide
- ✅ **Custom Exception Handling**: Standardized error handling with structured logging
- ✅ **Performance Awareness**: Existing caching service and performance monitoring infrastructure
- ✅ **Modern Django Stack**: Current dependencies include `djangorestframework`, `django-redis`, `sentry-sdk`
### Current Implementations
```python
# Existing API Pattern (parks/api/views.py)
class ParkApi(CreateApiMixin, UpdateApiMixin, ListApiMixin, RetrieveApiMixin, DestroyApiMixin, GenericViewSet):
InputSerializer = ParkCreateInputSerializer
OutputSerializer = ParkDetailOutputSerializer
FilterSerializer = ParkFilterInputSerializer
```
```python
# Existing Cache Service (core/services/map_cache_service.py)
class MapCacheService:
DEFAULT_TTL = 3600 # 1 hour
CLUSTER_TTL = 7200 # 2 hours
# Geographic partitioning with Redis
```
```python
# Existing Logging (core/logging.py)
def log_exception(logger, exception, *, context=None, request=None):
# Structured logging with context
```
## Priority 1: API Standardization
### 1.1 Nested Serializers Enhancement
**Current State**: Basic Input/Output serializer separation exists
**Goal**: Migrate to fully inline nested serializers
#### Implementation Plan
**Phase 1: Audit Current Serializers**
```bash
# Add to pyproject.toml dependencies (already exists)
"djangorestframework>=3.14.0"
```
**Phase 2: Enhance Nested Serializer Patterns**
```python
# Enhanced pattern for parks/api/serializers.py
class ParkCreateInputSerializer(serializers.Serializer):
class LocationInputSerializer(serializers.Serializer):
latitude = serializers.DecimalField(max_digits=9, decimal_places=6)
longitude = serializers.DecimalField(max_digits=9, decimal_places=6)
city = serializers.CharField(max_length=100)
state = serializers.CharField(max_length=100)
country = serializers.CharField(max_length=100)
class OperatorInputSerializer(serializers.Serializer):
name = serializers.CharField(max_length=200)
website = serializers.URLField(required=False)
name = serializers.CharField(max_length=200)
description = serializers.CharField(allow_blank=True)
location = LocationInputSerializer()
operator = OperatorInputSerializer(required=False)
opening_date = serializers.DateField(required=False)
```
**Implementation Tasks:**
1. **Enhance existing serializers** in `parks/api/serializers.py` and `rides/api/serializers.py`
2. **Create reusable nested serializers** for common patterns (Location, Company, etc.)
3. **Update API mixins** in `core/api/mixins.py` to handle nested validation
4. **Add serializer composition utilities** for complex nested structures
### 1.2 OpenAPI Documentation Implementation
**Recommended Module**: `drf-spectacular` (modern, actively maintained)
#### Implementation Plan
**Phase 1: Install and Configure**
```bash
# Add to pyproject.toml
"drf-spectacular>=0.27.0"
```
**Phase 2: Configuration**
```python
# config/django/base.py additions
INSTALLED_APPS = [
# ... existing apps
'drf_spectacular',
]
REST_FRAMEWORK = {
# ... existing settings
'DEFAULT_SCHEMA_CLASS': 'drf_spectacular.openapi.AutoSchema',
}
SPECTACULAR_SETTINGS = {
'TITLE': 'ThrillWiki API',
'DESCRIPTION': 'Comprehensive theme park and ride information API',
'VERSION': '1.0.0',
'SERVE_INCLUDE_SCHEMA': False,
'COMPONENT_SPLIT_REQUEST': True,
'TAGS': [
{'name': 'parks', 'description': 'Theme park operations'},
{'name': 'rides', 'description': 'Ride information and management'},
{'name': 'locations', 'description': 'Geographic location services'},
]
}
```
**Phase 3: URL Configuration**
```python
# thrillwiki/urls.py additions
from drf_spectacular.views import SpectacularAPIView, SpectacularSwaggerView, SpectacularRedocView
urlpatterns = [
# ... existing patterns
path('api/schema/', SpectacularAPIView.as_view(), name='schema'),
path('api/docs/', SpectacularSwaggerView.as_view(url_name='schema'), name='swagger-ui'),
path('api/redoc/', SpectacularRedocView.as_view(url_name='schema'), name='redoc'),
]
```
**Phase 4: Enhanced Documentation**
```python
# Enhanced API views with documentation
from drf_spectacular.utils import extend_schema, OpenApiParameter
class ParkApi(CreateApiMixin, UpdateApiMixin, ListApiMixin, RetrieveApiMixin, DestroyApiMixin, GenericViewSet):
@extend_schema(
summary="Create a new theme park",
description="Creates a new theme park with location and operator information",
tags=['parks'],
responses={201: ParkDetailOutputSerializer}
)
def create(self, request, *args, **kwargs):
return super().create(request, *args, **kwargs)
@extend_schema(
summary="List theme parks",
description="Retrieve a paginated list of theme parks with filtering options",
parameters=[
OpenApiParameter(name='search', description='Search parks by name', type=str),
OpenApiParameter(name='country', description='Filter by country', type=str),
],
tags=['parks']
)
def list(self, request, *args, **kwargs):
return super().list(request, *args, **kwargs)
```
### 1.3 API Versioning Strategy Enhancement
**Current State**: Basic URL-based routing exists
**Goal**: Comprehensive versioning with backward compatibility
#### Implementation Plan
**Phase 1: Configure DRF Versioning**
```python
# config/django/base.py
REST_FRAMEWORK = {
# ... existing settings
'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
'ALLOWED_VERSIONS': ['v1', 'v2'],
'DEFAULT_VERSION': 'v1',
'VERSION_PARAM': 'version'
}
```
**Phase 2: Versioned URL Structure**
```python
# New structure for API URLs
# thrillwiki/urls.py
urlpatterns = [
# ... existing patterns
path('api/v1/', include('core.urls.api_v1', namespace='api-v1')),
path('api/v2/', include('core.urls.api_v2', namespace='api-v2')), # Future version
]
# core/urls/api_v1.py
from django.urls import path, include
urlpatterns = [
path('parks/', include('parks.api.urls')),
path('rides/', include('rides.api.urls')),
path('locations/', include('location.api.urls')),
]
```
**Phase 3: Version-Aware Serializers**
```python
# Enhanced API mixins with versioning support
class VersionedApiMixin:
def get_serializer_class(self):
version = getattr(self.request, 'version', 'v1')
serializer_name = f"{self.__class__.__name__.replace('Api', '')}Serializer_v{version}"
# Fallback to default if version-specific serializer doesn't exist
try:
return getattr(self, serializer_name, self.serializer_class)
except AttributeError:
return self.serializer_class
```
## Priority 2: Performance Enhancement
### 2.1 Redis Caching Strategy Implementation
**Current State**: `django-redis` already in dependencies, `MapCacheService` exists
**Goal**: Comprehensive multi-layer caching strategy
#### Implementation Plan
**Phase 1: Enhanced Redis Configuration**
```python
# config/django/base.py enhancement
CACHES = {
'default': {
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': env('REDIS_URL', default='redis://127.0.0.1:6379/1'),
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
'PARSER_CLASS': 'redis.connection.HiredisParser',
'CONNECTION_POOL_CLASS': 'redis.BlockingConnectionPool',
'CONNECTION_POOL_CLASS_KWARGS': {
'max_connections': 50,
'timeout': 20,
},
'COMPRESSOR': 'django_redis.compressors.zlib.ZlibCompressor',
'IGNORE_EXCEPTIONS': True,
},
'KEY_PREFIX': 'thrillwiki',
'VERSION': 1,
},
'sessions': {
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': env('REDIS_URL', default='redis://127.0.0.1:6379/2'),
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
}
},
'api': {
'BACKEND': 'django_redis.cache.RedisCache',
'LOCATION': env('REDIS_URL', default='redis://127.0.0.1:6379/3'),
'OPTIONS': {
'CLIENT_CLASS': 'django_redis.client.DefaultClient',
}
}
}
# Use Redis for sessions
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
SESSION_CACHE_ALIAS = 'sessions'
SESSION_COOKIE_AGE = 86400 # 24 hours
```
**Phase 2: Enhanced Caching Service**
```python
# core/services/enhanced_cache_service.py
from typing import Optional, Any, Dict, List
from django.core.cache import caches
from django.core.cache.utils import make_template_fragment_key
import hashlib
import json
class EnhancedCacheService:
"""Comprehensive caching service with multiple cache backends"""
def __init__(self):
self.default_cache = caches['default']
self.api_cache = caches['api']
# L1: Query-level caching
def cache_queryset(self, cache_key: str, queryset_func, timeout: int = 3600, **kwargs):
"""Cache expensive querysets"""
cached_result = self.default_cache.get(cache_key)
if cached_result is None:
result = queryset_func(**kwargs)
self.default_cache.set(cache_key, result, timeout)
return result
return cached_result
# L2: API response caching
def cache_api_response(self, view_name: str, params: Dict, response_data: Any, timeout: int = 1800):
"""Cache API responses based on view and parameters"""
cache_key = self._generate_api_cache_key(view_name, params)
self.api_cache.set(cache_key, response_data, timeout)
def get_cached_api_response(self, view_name: str, params: Dict) -> Optional[Any]:
"""Retrieve cached API response"""
cache_key = self._generate_api_cache_key(view_name, params)
return self.api_cache.get(cache_key)
# L3: Geographic caching (building on existing MapCacheService)
def cache_geographic_data(self, bounds: 'GeoBounds', data: Any, zoom_level: int, timeout: int = 1800):
"""Cache geographic data with spatial keys"""
# Leverage existing MapCacheService implementation
pass
def _generate_api_cache_key(self, view_name: str, params: Dict) -> str:
"""Generate consistent cache keys for API responses"""
params_str = json.dumps(params, sort_keys=True)
params_hash = hashlib.md5(params_str.encode()).hexdigest()
return f"api:{view_name}:{params_hash}"
```
**Phase 3: Caching Decorators and Mixins**
```python
# core/decorators/cache_decorators.py
from functools import wraps
from django.core.cache import cache
def cache_api_response(timeout=1800, vary_on=None):
"""Decorator for caching API responses"""
def decorator(view_func):
@wraps(view_func)
def wrapper(self, request, *args, **kwargs):
if request.method != 'GET':
return view_func(self, request, *args, **kwargs)
# Generate cache key based on view, user, and parameters
cache_key_parts = [
view_func.__name__,
str(request.user.id) if request.user.is_authenticated else 'anonymous',
str(hash(frozenset(request.GET.items())))
]
if vary_on:
for field in vary_on:
cache_key_parts.append(str(getattr(request, field, '')))
cache_key = ':'.join(cache_key_parts)
# Try to get from cache
cached_response = cache.get(cache_key)
if cached_response:
return cached_response
# Execute view and cache result
response = view_func(self, request, *args, **kwargs)
if response.status_code == 200:
cache.set(cache_key, response, timeout)
return response
return wrapper
return decorator
# Usage in API views
class ParkApi(GenericViewSet):
@cache_api_response(timeout=3600, vary_on=['version'])
def list(self, request, *args, **kwargs):
return super().list(request, *args, **kwargs)
```
### 2.2 Database Optimization and Query Monitoring
**Recommended Modules**: `django-silk` (comprehensive), `django-debug-toolbar` (development)
#### Implementation Plan
**Phase 1: Install Monitoring Tools**
```bash
# Add to pyproject.toml
"django-silk>=5.0.0"
"django-debug-toolbar>=4.0.0" # Development only
"nplusone>=1.0.0" # N+1 query detection
```
**Phase 2: Configuration**
```python
# config/django/local.py (development)
INSTALLED_APPS = [
# ... existing apps
'silk',
'debug_toolbar',
'nplusone.ext.django',
]
MIDDLEWARE = [
'silk.middleware.SilkyMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware',
'nplusone.ext.django.NPlusOneMiddleware',
# ... existing middleware
]
# Silk configuration
SILKY_PYTHON_PROFILER = True
SILKY_PYTHON_PROFILER_BINARY = True
SILKY_PYTHON_PROFILER_RESULT_PATH = BASE_DIR / 'profiles'
# Debug toolbar configuration
INTERNAL_IPS = ['127.0.0.1', '::1']
# NPlusOne configuration
NPLUSONE_LOGGER = logging.getLogger('nplusone')
NPLUSONE_LOG_LEVEL = logging.WARN
```
**Phase 3: Query Optimization Utilities**
```python
# core/utils/query_optimization.py
from django.db import connection
from django.conf import settings
import logging
import time
from contextlib import contextmanager
logger = logging.getLogger('query_optimization')
@contextmanager
def track_queries(operation_name: str):
"""Context manager to track database queries for specific operations"""
if not settings.DEBUG:
yield
return
initial_queries = len(connection.queries)
start_time = time.time()
try:
yield
finally:
end_time = time.time()
total_queries = len(connection.queries) - initial_queries
execution_time = end_time - start_time
if total_queries > 10 or execution_time > 1.0:
logger.warning(
f"Performance concern in {operation_name}: "
f"{total_queries} queries, {execution_time:.2f}s"
)
# Enhanced selector patterns with query optimization
def park_list_optimized(*, filters: Optional[Dict] = None) -> QuerySet:
"""Optimized park list query with proper select_related and prefetch_related"""
queryset = Park.objects.select_related(
'location',
'operator',
'created_by'
).prefetch_related(
'areas',
'rides__manufacturer',
'reviews__user'
).annotate(
ride_count=Count('rides'),
average_rating=Avg('reviews__rating'),
latest_review_date=Max('reviews__created_at')
)
if filters:
queryset = queryset.filter(**filters)
return queryset.order_by('name')
```
**Phase 4: Database Index Optimization**
```python
# Enhanced model indexes based on common queries
class Park(TimeStampedModel):
class Meta:
indexes = [
models.Index(fields=['slug']),
models.Index(fields=['status', 'created_at']),
models.Index(fields=['location', 'status']),
models.Index(fields=['operator', 'status']),
models.Index(fields=['-average_rating', 'status']), # For top-rated parks
models.Index(fields=['opening_date', 'status']), # For chronological queries
]
# Add database-level constraints
constraints = [
models.CheckConstraint(
check=models.Q(average_rating__gte=0) & models.Q(average_rating__lte=5),
name='valid_rating_range'
),
]
```
### 2.3 Cloudflare Images CDN Integration
**Current State**: WhiteNoise for static files, local media storage
**Goal**: Cloudflare Images for media optimization and delivery, WhiteNoise for static files
[Cloudflare Images](https://developers.cloudflare.com/images/) provides an end-to-end solution for image storage, transformation, and delivery on Cloudflare's global network. This is ideal for ThrillWiki's image-heavy content (park photos, ride images, user submissions).
#### Implementation Plan
**Phase 1: Enhanced Static File Configuration**
```python
# config/django/production.py
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
# Enhanced WhiteNoise configuration for static files (CSS, JS)
WHITENOISE_USE_FINDERS = True
WHITENOISE_AUTOREFRESH = False
WHITENOISE_MAX_AGE = 31536000 # 1 year
WHITENOISE_SKIP_COMPRESS_EXTENSIONS = ['webp', 'avif']
# Static file optimization
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]
```
**Phase 2: Cloudflare Images Integration with django-cloudflare-images**
```bash
# Add to pyproject.toml - Use the official django-cloudflare-images package
"django-cloudflare-images>=0.6.0" # Latest version as of May 2024
```
```python
# config/django/base.py - Cloudflare Images configuration
# Using django-cloudflare-images package for simplified integration
# Storage configuration (Django 4.2+)
STORAGES = {
"default": {
"BACKEND": "cloudflare_images.storage.CloudflareImagesStorage"
},
"staticfiles": {
"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage"
}
}
# For Django < 4.2 (fallback)
DEFAULT_FILE_STORAGE = "cloudflare_images.storage.CloudflareImagesStorage"
# Cloudflare Images configuration
CLOUDFLARE_IMAGES_ACCOUNT_ID = env('CLOUDFLARE_IMAGES_ACCOUNT_ID')
CLOUDFLARE_IMAGES_API_TOKEN = env('CLOUDFLARE_IMAGES_API_TOKEN') # Images:Edit permission
CLOUDFLARE_IMAGES_ACCOUNT_HASH = env('CLOUDFLARE_IMAGES_ACCOUNT_HASH')
# Optional: Custom domain for image delivery
CLOUDFLARE_IMAGES_DOMAIN = env('CLOUDFLARE_IMAGES_DOMAIN', default=None) # e.g., "images.thrillwiki.com"
# Optional: Default variant for serving images
CLOUDFLARE_IMAGES_VARIANT = env('CLOUDFLARE_IMAGES_VARIANT', default='public')
# Optional: API timeout override
CLOUDFLARE_IMAGES_API_TIMEOUT = env('CLOUDFLARE_IMAGES_API_TIMEOUT', default=60, cast=int)
```
**Phase 3: Enhanced Model Fields with CloudflareImagesField**
```python
# parks/models/parks.py - Enhanced with CloudflareImagesField
from cloudflare_images.field import CloudflareImagesField
from django.db import models
class Park(TimeStampedModel):
# ... existing fields ...
# Replace ImageField with CloudflareImagesField for variant support
featured_image = CloudflareImagesField(
variant="hero", # Use 'hero' variant by default for park featured images
upload_to='parks/',
blank=True,
null=True,
help_text="Main park image displayed on detail pages"
)
# Additional image fields with specific variants
thumbnail_image = CloudflareImagesField(
variant="thumbnail",
upload_to='parks/thumbnails/',
blank=True,
null=True,
help_text="Thumbnail image for park listings"
)
# rides/models/rides.py - Enhanced ride images
class Ride(TimeStampedModel):
# ... existing fields ...
main_image = CloudflareImagesField(
variant="large",
upload_to='rides/',
blank=True,
null=True,
help_text="Primary ride image"
)
gallery_images = models.ManyToManyField(
'media.RideImage',
blank=True,
related_name='rides',
help_text="Additional ride photos"
)
# media/models.py - Gallery and user upload models
class RideImage(TimeStampedModel):
"""Individual ride images for galleries"""
image = CloudflareImagesField(
variant="medium",
upload_to='rides/gallery/',
help_text="Ride gallery image"
)
caption = models.CharField(max_length=200, blank=True)
photographer = models.CharField(max_length=100, blank=True)
is_approved = models.BooleanField(default=False)
class UserSubmission(TimeStampedModel):
"""User-submitted images for moderation"""
image = CloudflareImagesField(
variant="public", # Use public variant for moderation workflow
upload_to='submissions/',
help_text="User-submitted image awaiting moderation"
)
submitted_by = models.ForeignKey('accounts.User', on_delete=models.CASCADE)
content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
is_approved = models.BooleanField(default=False)
moderation_notes = models.TextField(blank=True)
```
**Phase 4: Enhanced Cloudflare Images Service Layer**
```python
# media/services/cloudflare_optimization.py
from django.conf import settings
from typing import Dict, List, Optional
import logging
logger = logging.getLogger(__name__)
class CloudflareImagesService:
"""
Enhanced service for Cloudflare Images operations
Works with django-cloudflare-images package
"""
def __init__(self):
self.account_hash = settings.CLOUDFLARE_IMAGES_ACCOUNT_HASH
self.domain = getattr(settings, 'CLOUDFLARE_IMAGES_DOMAIN', None)
self.base_url = f"https://{self.domain}" if self.domain else "https://imagedelivery.net"
def get_image_url(self, image_id: str, variant: str = 'public', **transforms) -> str:
"""
Generate Cloudflare Images URL with optional transformations
Args:
image_id: Cloudflare image ID (from CloudflareImagesField)
variant: Predefined variant or 'public' for custom transforms
transforms: Custom transformation parameters (width, height, fit, format, etc.)
"""
if not image_id:
return ''
if transforms:
# Build custom transformation string
transform_parts = []
for key, value in transforms.items():
transform_parts.append(f"{key}={value}")
variant = ','.join(transform_parts)
return f"{self.base_url}/{self.account_hash}/{image_id}/{variant}"
def get_responsive_urls(self, image_id: str) -> Dict[str, str]:
"""
Generate responsive image URLs for different screen sizes
Uses Cloudflare's automatic optimization and format selection
"""
if not image_id:
return {}
return {
# Standard variants for different use cases
'thumbnail': self.get_image_url(image_id, width=150, height=150, fit='cover'),
'small': self.get_image_url(image_id, width=300, height=300, fit='cover'),
'medium': self.get_image_url(image_id, width=600, height=600, fit='cover'),
'large': self.get_image_url(image_id, width=1200, height=1200, fit='cover'),
'hero': self.get_image_url(image_id, width=1920, height=1080, fit='cover'),
# WebP variants for modern browsers
'webp_small': self.get_image_url(image_id, width=300, height=300, fit='cover', format='webp'),
'webp_medium': self.get_image_url(image_id, width=600, height=600, fit='cover', format='webp'),
'webp_large': self.get_image_url(image_id, width=1200, height=1200, fit='cover', format='webp'),
# AVIF for ultra-modern browsers
'avif_medium': self.get_image_url(image_id, width=600, height=600, fit='cover', format='avif'),
# Original (Cloudflare will still optimize based on request headers)
'original': self.get_image_url(image_id, 'public'),
}
def get_srcset_string(self, image_id: str, sizes: List[int] = None) -> str:
"""
Generate srcset string for responsive images
Args:
image_id: Cloudflare image ID
sizes: List of widths for srcset (defaults to common breakpoints)
"""
if not image_id:
return ''
sizes = sizes or [320, 640, 768, 1024, 1280, 1536, 1920]
srcset_parts = []
for width in sizes:
url = self.get_image_url(image_id, width=width, fit='cover')
srcset_parts.append(f"{url} {width}w")
return ', '.join(srcset_parts)
def optimize_for_context(self, image_id: str, context: str = 'default') -> str:
"""
Get optimized image URL based on usage context
Args:
image_id: Cloudflare image ID
context: Usage context (hero, card, thumbnail, avatar, etc.)
"""
context_configs = {
'hero': {'width': 1920, 'height': 1080, 'fit': 'cover', 'quality': 85},
'card': {'width': 400, 'height': 300, 'fit': 'cover', 'quality': 80},
'thumbnail': {'width': 150, 'height': 150, 'fit': 'cover', 'quality': 75},
'avatar': {'width': 100, 'height': 100, 'fit': 'cover', 'quality': 80},
'gallery': {'width': 800, 'height': 600, 'fit': 'cover', 'quality': 85},
'list_item': {'width': 300, 'height': 200, 'fit': 'cover', 'quality': 75},
}
config = context_configs.get(context, {'width': 600, 'height': 400, 'fit': 'cover'})
return self.get_image_url(image_id, **config)
# Template integration helpers
class CloudflareImagesTemplateService:
"""Enhanced template integration for Cloudflare Images"""
@staticmethod
def get_picture_element(image_id: str, alt_text: str = '', css_classes: str = '',
context: str = 'default') -> str:
"""
Generate modern picture element with format-based source selection
Provides AVIF, WebP, and fallback support
"""
if not image_id:
return f''
service = CloudflareImagesService()
urls = service.get_responsive_urls(image_id)
srcset = service.get_srcset_string(image_id)
return f"""
"""
@staticmethod
def get_responsive_img(image_id: str, alt_text: str = '', css_classes: str = '',
context: str = 'default') -> str:
"""
Generate responsive img element with srcset
Simpler alternative to picture element
"""
if not image_id:
return f''
service = CloudflareImagesService()
srcset = service.get_srcset_string(image_id)
fallback_url = service.optimize_for_context(image_id, context)
return f"""
"""
```
**Phase 5: Enhanced Django Template Integration**
```python
# media/templatetags/cloudflare_images.py
from django import template
from django.utils.safestring import mark_safe
from media.services.cloudflare_optimization import CloudflareImagesService, CloudflareImagesTemplateService
register = template.Library()
@register.simple_tag
def cf_image_url(image_field, **transforms):
"""
Get Cloudflare Images URL with optional transformations
Works with CloudflareImagesField instances
"""
if not image_field:
return ''
# Extract image ID from CloudflareImagesField
image_id = str(image_field) if image_field else ''
service = CloudflareImagesService()
if transforms:
return service.get_image_url(image_id, **transforms)
else:
# Use the field's default variant if no transforms specified
variant = getattr(image_field.field, 'variant', 'public')
return service.get_image_url(image_id, variant)
@register.simple_tag
def cf_responsive_image(image_field, alt_text='', css_classes='', context='default'):
"""Generate responsive picture element with modern format support"""
if not image_field:
return mark_safe(f'')
image_id = str(image_field) if image_field else ''
return mark_safe(CloudflareImagesTemplateService.get_picture_element(
image_id, alt_text, css_classes, context
))
@register.simple_tag
def cf_img_responsive(image_field, alt_text='', css_classes='', context='default'):
"""Generate responsive img element with srcset (simpler alternative)"""
if not image_field:
return mark_safe(f'')
image_id = str(image_field) if image_field else ''
return mark_safe(CloudflareImagesTemplateService.get_responsive_img(
image_id, alt_text, css_classes, context
))
@register.simple_tag
def cf_optimize(image_field, context='default'):
"""Get context-optimized image URL"""
if not image_field:
return ''
image_id = str(image_field) if image_field else ''
service = CloudflareImagesService()
return service.optimize_for_context(image_id, context)
@register.simple_tag
def cf_srcset(image_field, sizes=None):
"""Generate srcset string for responsive images"""
if not image_field:
return ''
image_id = str(image_field) if image_field else ''
service = CloudflareImagesService()
if sizes:
# Convert comma-separated string to list if needed
if isinstance(sizes, str):
sizes = [int(s.strip()) for s in sizes.split(',')]
return service.get_srcset_string(image_id, sizes)
else:
return service.get_srcset_string(image_id)
@register.inclusion_tag('components/cloudflare_image.html')
def cf_image_component(image_field, alt_text='', css_classes='', context='default',
show_caption=False, caption=''):
"""
Render a complete image component with optional caption
Uses inclusion tag for complex HTML structure
"""
return {
'image_field': image_field,
'alt_text': alt_text,
'css_classes': css_classes,
'context': context,
'show_caption': show_caption,
'caption': caption,
}
```
**Template Component (components/cloudflare_image.html):**
```html
{% load cloudflare_images %}
{% if image_field %}
{% cf_responsive_image image_field alt_text "w-full h-auto" context %}
{% if show_caption and caption %}
{{ caption }}
{% endif %}
{% else %}
No image available
{% endif %}
```
**Enhanced Usage in Templates:**
```html
{% load cloudflare_images %}
{% cf_responsive_image park.featured_image park.name "w-full h-64 object-cover" "hero" %}
{% cf_img_responsive ride.main_image ride.name "rounded-lg" "card" %}
{% cf_image_component ride.main_image ride.name "gallery-image" "gallery" True "Photo taken in 2024" %}
```
**Migration Script for Existing ImageFields:**
```python
# management/commands/migrate_to_cloudflare_images.py
from django.core.management.base import BaseCommand
from django.apps import apps
from parks.models import Park
from rides.models import Ride
import requests
import logging
logger = logging.getLogger(__name__)
class Command(BaseCommand):
help = 'Migrate existing ImageField files to Cloudflare Images'
def add_arguments(self, parser):
parser.add_argument('--dry-run', action='store_true', help='Show what would be migrated without doing it')
parser.add_argument('--model', type=str, help='Specific model to migrate (e.g., parks.Park)')
def handle(self, *args, **options):
dry_run = options['dry_run']
specific_model = options.get('model')
models_to_migrate = []
if specific_model:
app_label, model_name = specific_model.split('.')
models_to_migrate.append(apps.get_model(app_label, model_name))
else:
models_to_migrate = [Park, Ride] # Add other models as needed
for model in models_to_migrate:
self.migrate_model(model, dry_run)
def migrate_model(self, model, dry_run=False):
"""Migrate a specific model's ImageFields to CloudflareImagesFields"""
self.stdout.write(f"Processing {model.__name__}...")
# Get all instances with images
instances = model.objects.exclude(featured_image='').exclude(featured_image=None)
for instance in instances:
if instance.featured_image:
if dry_run:
self.stdout.write(f"Would migrate: {instance} - {instance.featured_image.url}")
else:
self.migrate_image_field(instance, 'featured_image')
def migrate_image_field(self, instance, field_name):
"""Migrate a specific image field to Cloudflare Images"""
try:
field = getattr(instance, field_name)
if field and hasattr(field, 'url'):
# The django-cloudflare-images package will handle the upload
# when you save the instance with the new CloudflareImagesField
self.stdout.write(f"Migrated: {instance} - {field_name}")
except Exception as e:
logger.error(f"Failed to migrate {instance} - {field_name}: {e}")
```
## Priority 3: Monitoring & Observability
### 3.1 Error Tracking with Sentry Integration
**Current State**: `sentry-sdk` already in dependencies, basic logging exists
**Goal**: Comprehensive error tracking with performance monitoring
#### Implementation Plan
**Phase 1: Enhanced Sentry Configuration**
```python
# config/django/base.py
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.redis import RedisIntegration
from sentry_sdk.integrations.logging import LoggingIntegration
# Sentry logging integration
sentry_logging = LoggingIntegration(
level=logging.INFO, # Capture info and above as breadcrumbs
event_level=logging.ERROR # Send records as events
)
sentry_sdk.init(
dsn=env('SENTRY_DSN', default=''),
integrations=[
DjangoIntegration(
transaction_style='url',
middleware_spans=True,
signals_spans=True,
cache_spans=True,
),
RedisIntegration(),
sentry_logging,
],
traces_sample_rate=env('SENTRY_TRACES_SAMPLE_RATE', default=0.1, cast=float),
profiles_sample_rate=env('SENTRY_PROFILES_SAMPLE_RATE', default=0.1, cast=float),
send_default_pii=False,
environment=env('DJANGO_ENV', default='development'),
before_send=sentry_filter_errors,
)
def sentry_filter_errors(event, hint):
"""Filter out common non-critical errors"""
if 'exc_info' in hint:
exc_type, exc_value, tb = hint['exc_info']
if isinstance(exc_value, (Http404, PermissionDenied)):
return None
return event
```
**Phase 2: Enhanced Error Context**
```python
# core/middleware/sentry_middleware.py
from sentry_sdk import set_user, set_tag, set_context
class SentryContextMiddleware:
"""Add context to Sentry errors"""
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
# Set user context
if hasattr(request, 'user') and request.user.is_authenticated:
set_user({
'id': request.user.id,
'username': request.user.username,
'email': request.user.email,
})
# Set request context
set_context('request', {
'url': request.build_absolute_uri(),
'method': request.method,
'headers': dict(request.headers),
})
# Set custom tags
set_tag('user_agent', request.META.get('HTTP_USER_AGENT', ''))
set_tag('ip_address', self._get_client_ip(request))
response = self.get_response(request)
return response
def _get_client_ip(self, request):
x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
if x_forwarded_for:
return x_forwarded_for.split(',')[0]
return request.META.get('REMOTE_ADDR')
```
**Phase 3: Custom Performance Monitoring**
```python
# core/services/performance_monitoring.py
import time
from contextlib import contextmanager
from sentry_sdk import start_transaction, capture_message
import logging
logger = logging.getLogger(__name__)
@contextmanager
def monitor_performance(operation_name: str, **tags):
"""Context manager for monitoring operation performance"""
with start_transaction(op=operation_name, name=operation_name) as transaction:
# Set tags
for key, value in tags.items():
transaction.set_tag(key, value)
start_time = time.time()
try:
yield transaction
finally:
duration = time.time() - start_time
transaction.set_data('duration_seconds', duration)
# Log slow operations
if duration > 2.0: # Log operations slower than 2 seconds
capture_message(
f"Slow operation detected: {operation_name}",
level='warning'
)
# Usage in services
class ParkService:
@classmethod
def create_park(cls, **park_data):
with monitor_performance('park_creation', category='parks'):
# Park creation logic
pass
```
### 3.2 Application Performance Monitoring (APM) Integration
**Recommended Approach**: Enhance Sentry APM + Custom Metrics
#### Implementation Plan
**Phase 1: Enhanced Django Logging**
```python
# config/django/base.py - Enhanced logging configuration
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'verbose': {
'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
'style': '{',
},
'json': {
'()': 'pythonjsonlogger.jsonlogger.JsonFormatter',
'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'verbose',
},
'file': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': BASE_DIR / 'logs' / 'thrillwiki.log',
'maxBytes': 1024*1024*10, # 10MB
'backupCount': 5,
'formatter': 'json',
},
'performance': {
'class': 'logging.handlers.RotatingFileHandler',
'filename': BASE_DIR / 'logs' / 'performance.log',
'maxBytes': 1024*1024*10, # 10MB
'backupCount': 5,
'formatter': 'json',
},
},
'root': {
'level': 'INFO',
'handlers': ['console'],
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'INFO',
'propagate': False,
},
'thrillwiki': {
'handlers': ['file'],
'level': 'INFO',
'propagate': False,
},
'performance': {
'handlers': ['performance'],
'level': 'INFO',
'propagate': False,
},
'query_optimization': {
'handlers': ['file'],
'level': 'WARNING',
'propagate': False,
},
},
}
```
**Phase 2: Performance Metrics Collection**
```python
# core/middleware/performance_middleware.py
import time
import logging
from django.db import connection
performance_logger = logging.getLogger('performance')
class PerformanceMiddleware:
"""Middleware to collect performance metrics"""
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
start_time = time.time()
initial_queries = len(connection.queries)
response = self.get_response(request)
# Calculate metrics
duration = time.time() - start_time
queries_count = len(connection.queries) - initial_queries
# Log performance data
performance_data = {
'path': request.path,
'method': request.method,
'status_code': response.status_code,
'duration_ms': round(duration * 1000, 2),
'queries_count': queries_count,
'content_length': len(response.content) if hasattr(response, 'content') else 0,
'user_id': getattr(request.user, 'id', None) if hasattr(request, 'user') else None,
}
performance_logger.info('request_performance', extra=performance_data)
# Add performance headers for debugging
if hasattr(response, '__setitem__'):
response['X-Response-Time'] = f"{duration * 1000:.2f}ms"
response['X-Query-Count'] = str(queries_count)
return response
```
### 3.3 Comprehensive Health Checks Implementation
**Recommended Module**: `django-health-check` (already good foundation)
#### Implementation Plan
**Phase 1: Install and Configure Health Checks**
```bash
# Add to pyproject.toml
"django-health-check>=3.17.0"
```
**Phase 2: Comprehensive Health Check Configuration**
```python
# config/django/base.py
INSTALLED_APPS = [
# ... existing apps
'health_check',
'health_check.db',
'health_check.cache',
'health_check.storage',
'health_check.contrib.migrations',
'health_check.contrib.redis',
]
HEALTH_CHECK = {
'DISK_USAGE_MAX': 90, # Fail if disk usage is over 90%
'MEMORY_MIN': 100, # Fail if less than 100MB available memory
}
```
**Phase 3: Custom Health Checks**
```python
# core/health_checks/custom_checks.py
from health_check.backends import BaseHealthCheckBackend
from health_check.exceptions import ServiceUnavailable
from django.core.cache import cache
from django.db import connection
import redis
class CacheHealthCheck(BaseHealthCheckBackend):
"""Check Redis cache connectivity and performance"""
critical_service = True
def check_status(self):
try:
# Test cache write/read
test_key = 'health_check_test'
test_value = 'test_value'
cache.set(test_key, test_value, timeout=30)
retrieved_value = cache.get(test_key)
if retrieved_value != test_value:
self.add_error("Cache read/write test failed")
cache.delete(test_key)
except Exception as e:
self.add_error(f"Cache service unavailable: {e}")
class DatabasePerformanceCheck(BaseHealthCheckBackend):
"""Check database performance"""
critical_service = False
def check_status(self):
try:
import time
start_time = time.time()
with connection.cursor() as cursor:
cursor.execute("SELECT 1")
result = cursor.fetchone()
query_time = time.time() - start_time
if query_time > 1.0: # Warn if query takes more than 1 second
self.add_error(f"Database responding slowly: {query_time:.2f}s")
except Exception as e:
self.add_error(f"Database performance check failed: {e}")
class ExternalServiceHealthCheck(BaseHealthCheckBackend):
"""Check external services (APIs, etc.)"""
critical_service = False
def check_status(self):
# Check external dependencies
# (e.g., geocoding services, email services)
pass
# Register custom health checks
# config/django/base.py
HEALTH_CHECK_BACKENDS = [
'health_check.db',
'health_check.cache',
'health_check.storage',
'core.health_checks.custom_checks.CacheHealthCheck',
'core.health_checks.custom_checks.DatabasePerformanceCheck',
'core.health_checks.custom_checks.ExternalServiceHealthCheck',
]
```
**Phase 4: Health Check Endpoints**
```python
# thrillwiki/urls.py additions
urlpatterns = [
# ... existing patterns
path('health/', include('health_check.urls')),
path('health/api/', HealthCheckAPIView.as_view(), name='health-api'),
]
# core/views/health_views.py
from rest_framework.views import APIView
from rest_framework.response import Response
from health_check.views import MainView
import json
class HealthCheckAPIView(APIView):
"""API endpoint for health checks with JSON response"""
permission_classes = [] # Public endpoint
def get(self, request):
# Get health check results
main_view = MainView()
main_view.request = request
plugins = main_view.plugins
errors = main_view.errors
# Format response
health_data = {
'status': 'healthy' if not errors else 'unhealthy',
'timestamp': timezone.now().isoformat(),
'checks': {}
}
for plugin in plugins:
plugin_errors = errors.get(plugin.__class__.__name__, [])
health_data['checks'][plugin.identifier()] = {
'status': 'healthy' if not plugin_errors else 'unhealthy',
'errors': [str(error) for error in plugin_errors]
}
status_code = 200 if not errors else 503
return Response(health_data, status=status_code)
```
## Implementation Timeline and Phases
### Phase 1: Foundation (Weeks 1-2)
1. **API Documentation Setup**
- Install and configure `drf-spectacular`
- Add basic OpenAPI documentation to existing APIs
- Set up API versioning structure
2. **Monitoring Foundation**
- Enhance Sentry configuration
- Set up basic health checks
- Configure enhanced logging
### Phase 2: Performance Core (Weeks 3-4)
1. **Caching Enhancement**
- Implement multi-layer Redis caching
- Add caching decorators and mixins
- Optimize existing cache service
2. **Database Monitoring**
- Install and configure `django-silk`
- Add query optimization utilities
- Implement database indexes
### Phase 3: Advanced Features (Weeks 5-6)
1. **Nested Serializers Migration**
- Refactor existing serializers to inline patterns
- Add validation enhancements
- Update API documentation
2. **CDN Integration**
- Implement media optimization
- Set up responsive image serving
- Configure CDN fallbacks
### Phase 4: Monitoring & Observability (Weeks 7-8)
1. **Comprehensive Monitoring**
- Custom performance monitoring
- Advanced error tracking
- Health check expansion
2. **Testing and Optimization**
- Performance testing
- Load testing
- Final optimizations
## Success Metrics
### API Standardization
- ✅ 100% API endpoints documented with OpenAPI
- ✅ Consistent nested serializer patterns across all APIs
- ✅ Versioning strategy supporting backward compatibility
### Performance Enhancement
- 🎯 **Response Times**: API responses < 200ms (95th percentile)
- 🎯 **Cache Hit Rate**: > 80% for frequently accessed data
- 🎯 **Database Query Optimization**: < 10 queries per page load
### Monitoring & Observability
- 🎯 **Error Tracking**: 100% error capture with context
- 🎯 **Performance Monitoring**: Real-time performance metrics
- 🎯 **Health Checks**: Comprehensive system monitoring
## Risk Mitigation
### Technical Risks
1. **Cache Invalidation Complexity**
- Mitigation: Implement cache versioning and TTL strategies
- Fallback: Graceful degradation without cache
2. **CDN Configuration Issues**
- Mitigation: Local file serving fallback
- Testing: Comprehensive testing in staging environment
3. **Performance Monitoring Overhead**
- Mitigation: Configurable sampling rates
- Monitoring: Track monitoring overhead itself
### Operational Risks
1. **Deployment Complexity**
- Mitigation: Phased rollout with feature flags
- Rollback: Maintain ability to quickly revert changes
2. **Third-party Service Dependencies**
- Mitigation: Implement circuit breakers and fallbacks
- Monitoring: Health checks for external dependencies
## Conclusion
This comprehensive implementation plan leverages Django's robust ecosystem to enhance the ThrillWiki application across all three priority areas. The plan builds upon existing strengths while addressing current gaps, ensuring a scalable, observable, and high-performance application.
The phased approach allows for incremental improvements with immediate benefits, while the comprehensive monitoring ensures that performance gains are measurable and sustainable. Each enhancement is designed to work synergistically with others, creating a robust foundation for future development.
**Key Benefits:**
- 📈 **Improved Performance**: Multi-layer caching and database optimization
- 🔍 **Enhanced Observability**: Comprehensive monitoring and error tracking
- 📚 **Better Developer Experience**: Complete API documentation and tooling
- 🚀 **Scalability**: CDN integration and performance optimization
- 🛡️ **Reliability**: Health checks and error handling
This plan positions ThrillWiki for continued growth while maintaining code quality and operational excellence.