thrillwiki_django_no_react/docs/email-service.md

# Email Service Documentation

## Table of Contents

1. [Overview](#overview)
2. [Architecture](#architecture)
3. [Configuration](#configuration)
4. [API Usage](#api-usage)
5. [Django Email Backend](#django-email-backend)
6. [Testing](#testing)
7. [Management Commands](#management-commands)
8. [Troubleshooting](#troubleshooting)
9. [Best Practices](#best-practices)

## Overview

The Email Service is a comprehensive email delivery system built for the Django application. It provides a centralized way to send emails through the ForwardEmail API service, with support for site-specific configurations, Django email backend integration, and comprehensive testing tools.

### Key Features

- **Site-specific Configuration**: Different email settings per Django site
- **ForwardEmail Integration**: Uses ForwardEmail API for reliable email delivery
- **Django Backend Integration**: Drop-in replacement for Django's email backend
- **REST API Endpoint**: Send emails via HTTP API
- **Comprehensive Testing**: Built-in testing commands and flows
- **History Tracking**: All configurations are tracked with pghistory
- **Admin Interface**: Easy configuration management through Django admin

## Architecture

The email service consists of several key components:

```
apps/email_service/
├── models.py              # EmailConfiguration model
├── services.py            # Core EmailService class
├── backends.py            # Django email backend implementation
├── admin.py              # Django admin configuration
├── urls.py               # URL patterns (legacy)
└── management/commands/   # Testing and utility commands

apps/api/v1/email/
├── views.py              # Centralized API views
└── urls.py               # API URL patterns
```

### Component Overview

1. **EmailConfiguration Model**: Stores site-specific email settings
2. **EmailService**: Core service class for sending emails
3. **ForwardEmailBackend**: Django email backend implementation
4. **API Views**: REST API endpoints for email sending
5. **Management Commands**: Testing and utility tools

## Configuration

### Database Configuration

Email configurations are stored in the database and managed through the Django admin interface.

#### EmailConfiguration Model

The [`EmailConfiguration`](../backend/apps/email_service/models.py:8) model stores the following fields:

- `api_key`: ForwardEmail API key
- `from_email`: Default sender email address
- `from_name`: Display name for the sender
- `reply_to`: Reply-to email address
- `site`: Associated Django site (ForeignKey)
- `created_at`: Creation timestamp
- `updated_at`: Last update timestamp

#### Creating Configuration via Admin

1. Access Django admin at `/admin/`
2. Navigate to "Email Configurations"
3. Click "Add Email Configuration"
4. Fill in the required fields:
   - **Site**: Select the Django site
   - **API Key**: Your ForwardEmail API key
   - **From Name**: Display name (e.g., "ThrillWiki")
   - **From Email**: Sender email address
   - **Reply To**: Reply-to email address

### Environment Variables (Fallback)

If no database configuration exists, the system falls back to environment variables:

```bash
FORWARD_EMAIL_API_KEY=your_api_key_here
FORWARD_EMAIL_FROM=noreply@yourdomain.com
FORWARD_EMAIL_BASE_URL=https://api.forwardemail.net
```

### Django Settings

Add the email service to your Django settings:

```python
# settings.py
INSTALLED_APPS = [
    # ... other apps
    'apps.email_service',
]

# ForwardEmail API base URL
FORWARD_EMAIL_BASE_URL = 'https://api.forwardemail.net'

# Optional: Use custom email backend
EMAIL_BACKEND = 'apps.email_service.backends.ForwardEmailBackend'
```

## API Usage

### REST API Endpoint

The email service provides a REST API endpoint for sending emails.

#### Endpoint

```
POST /api/v1/email/send/
```

#### Request Format

```json
{
    "to": "recipient@example.com",
    "subject": "Email Subject",
    "text": "Email body text",
    "from_email": "sender@example.com"  // optional
}
```

#### Response Format

**Success (200 OK):**
```json
{
    "message": "Email sent successfully",
    "response": {
        // ForwardEmail API response
    }
}
```

**Error (400 Bad Request):**
```json
{
    "error": "Missing required fields",
    "required_fields": ["to", "subject", "text"]
}
```

**Error (500 Internal Server Error):**
```json
{
    "error": "Failed to send email: [error details]"
}
```

#### Example Usage

**cURL:**
```bash
curl -X POST http://localhost:8000/api/v1/email/send/ \
  -H "Content-Type: application/json" \
  -d '{
    "to": "user@example.com",
    "subject": "Welcome to ThrillWiki",
    "text": "Thank you for joining ThrillWiki!"
  }'
```

**JavaScript (fetch):**
```javascript
const response = await fetch('/api/v1/email/send/', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        'X-CSRFToken': getCookie('csrftoken')  // if CSRF protection enabled
    },
    body: JSON.stringify({
        to: 'user@example.com',
        subject: 'Welcome to ThrillWiki',
        text: 'Thank you for joining ThrillWiki!'
    })
});

const result = await response.json();
```

**Python (requests):**
```python
import requests

response = requests.post('http://localhost:8000/api/v1/email/send/', json={
    'to': 'user@example.com',
    'subject': 'Welcome to ThrillWiki',
    'text': 'Thank you for joining ThrillWiki!'
})

result = response.json()
```

### Direct Service Usage

You can also use the [`EmailService`](../backend/apps/email_service/services.py:11) class directly in your Python code:

```python
from django.contrib.sites.shortcuts import get_current_site
from apps.email_service.services import EmailService

# In a view or other code
def send_welcome_email(request, user_email):
    site = get_current_site(request)
    
    try:
        response = EmailService.send_email(
            to=user_email,
            subject="Welcome to ThrillWiki",
            text="Thank you for joining ThrillWiki!",
            site=site
        )
        return response
    except Exception as e:
        # Handle error
        print(f"Failed to send email: {e}")
        return None
```

## Django Email Backend

The email service includes a custom Django email backend that integrates with Django's built-in email system.

### Configuration

To use the custom backend, set it in your Django settings:

```python
# settings.py
EMAIL_BACKEND = 'apps.email_service.backends.ForwardEmailBackend'
```

### Usage

Once configured, you can use Django's standard email functions:

```python
from django.core.mail import send_mail
from django.contrib.sites.shortcuts import get_current_site

def send_notification(request):
    site = get_current_site(request)
    
    send_mail(
        subject='Notification',
        message='This is a notification email.',
        from_email=None,  # Will use site's default
        recipient_list=['user@example.com'],
        fail_silently=False,
    )
```

### Backend Features

The [`ForwardEmailBackend`](../backend/apps/email_service/backends.py:7) provides:

- **Site-aware Configuration**: Automatically uses the correct site's email settings
- **Error Handling**: Proper error handling with optional silent failures
- **Multiple Recipients**: Handles multiple recipients (sends individually)
- **From Email Handling**: Automatic from email resolution

### Backend Limitations

- **Single Recipient**: ForwardEmail API sends to one recipient at a time
- **Text Only**: Currently supports text emails only (HTML support can be added)
- **Site Requirement**: Requires site context for configuration lookup

## Testing

The email service includes comprehensive testing tools to verify functionality.

### Management Commands

#### Test Email Service

Test the core email service functionality:

```bash
cd backend && uv run manage.py test_email_service
```

**Options:**
- `--to EMAIL`: Recipient email (default: test@thrillwiki.com)
- `--api-key KEY`: Override API key
- `--from-email EMAIL`: Override from email

**Example:**
```bash
cd backend && uv run manage.py test_email_service --to user@example.com
```

This command tests:
1. Site configuration setup
2. Direct EmailService usage
3. API endpoint functionality
4. Django email backend

#### Test Email Flows

Test all application email flows:

```bash
cd backend && uv run manage.py test_email_flows
```

This command tests:
1. User registration emails
2. Password change notifications
3. Email change verification
4. Password reset emails

### Manual Testing

#### API Endpoint Testing

Test the API endpoint directly:

```bash
curl -X POST http://localhost:8000/api/v1/email/send/ \
  -H "Content-Type: application/json" \
  -d '{
    "to": "test@example.com",
    "subject": "Test Email",
    "text": "This is a test email."
  }'
```

#### Django Shell Testing

Test using Django shell:

```python
# cd backend && uv run manage.py shell
from django.contrib.sites.models import Site
from apps.email_service.services import EmailService

site = Site.objects.get_current()
response = EmailService.send_email(
    to="test@example.com",
    subject="Test from Shell",
    text="This is a test email from Django shell.",
    site=site
)
print(response)
```

## Management Commands

### test_email_service

**Purpose**: Comprehensive testing of email service functionality

**Location**: [`backend/apps/email_service/management/commands/test_email_service.py`](../backend/apps/email_service/management/commands/test_email_service.py:12)

**Usage:**
```bash
cd backend && uv run manage.py test_email_service [options]
```

**Options:**
- `--to EMAIL`: Recipient email address
- `--api-key KEY`: ForwardEmail API key override
- `--from-email EMAIL`: Sender email override

**Tests Performed:**
1. **Site Configuration**: Creates/updates email configuration
2. **Direct Service**: Tests EmailService.send_email()
3. **API Endpoint**: Tests REST API endpoint
4. **Django Backend**: Tests Django email backend

### test_email_flows

**Purpose**: Test all application email flows

**Location**: [`backend/apps/email_service/management/commands/test_email_flows.py`](../backend/apps/email_service/management/commands/test_email_flows.py:13)

**Usage:**
```bash
cd backend && uv run manage.py test_email_flows
```

**Tests Performed:**
1. **Registration**: User registration email flow
2. **Password Change**: Password change notification
3. **Email Change**: Email change verification
4. **Password Reset**: Password reset email flow

## Troubleshooting

### Common Issues

#### 1. "Email configuration is missing for site"

**Cause**: No EmailConfiguration exists for the current site.

**Solution:**
1. Access Django admin
2. Create EmailConfiguration for your site
3. Or set environment variables as fallback

#### 2. "Failed to send email (Status 401)"

**Cause**: Invalid or missing API key.

**Solution:**
1. Verify API key in EmailConfiguration
2. Check ForwardEmail account status
3. Ensure API key has proper permissions

#### 3. "Could not connect to server"

**Cause**: Django development server not running or wrong URL.

**Solution:**
1. Start Django server: `cd backend && uv run manage.py runserver`
2. Verify server is running on expected port
3. Check FORWARD_EMAIL_BASE_URL setting

#### 4. "Site matching query does not exist"

**Cause**: Default site not configured properly.

**Solution:**
```python
# Django shell
from django.contrib.sites.models import Site
Site.objects.get_or_create(
    id=1,
    defaults={'domain': 'localhost:8000', 'name': 'localhost:8000'}
)
```

### Debug Mode

The EmailService includes debug output. Check console logs for:
- Request URL and data
- Response status and body
- API key (masked)
- Site information

### Testing Configuration

Verify your configuration:

```bash
# Test with specific configuration
cd backend && uv run manage.py test_email_service \
  --api-key your_api_key \
  --from-email noreply@yourdomain.com \
  --to test@example.com
```

### API Response Codes

- **200**: Success
- **400**: Bad request (missing fields, invalid data)
- **401**: Unauthorized (invalid API key)
- **500**: Server error (configuration issues, network problems)

## Best Practices

### Security

1. **API Key Protection**: Store API keys securely, never in code
2. **Environment Variables**: Use environment variables for sensitive data
3. **HTTPS**: Always use HTTPS in production
4. **Rate Limiting**: Implement rate limiting for API endpoints

### Configuration Management

1. **Site-Specific Settings**: Use database configuration for multi-site setups
2. **Fallback Configuration**: Always provide environment variable fallbacks
3. **Admin Interface**: Use Django admin for easy configuration management
4. **Configuration Validation**: Test configurations after changes

### Error Handling

1. **Graceful Degradation**: Handle email failures gracefully
2. **Logging**: Log email failures for debugging
3. **User Feedback**: Provide appropriate user feedback
4. **Retry Logic**: Implement retry logic for transient failures

### Performance

1. **Async Processing**: Consider using Celery for email sending
2. **Batch Operations**: Group multiple emails when possible
3. **Connection Pooling**: Reuse HTTP connections when sending multiple emails
4. **Monitoring**: Monitor email delivery rates and failures

### Testing

1. **Automated Tests**: Include email testing in your test suite
2. **Test Environments**: Use test email addresses in development
3. **Integration Tests**: Test complete email flows
4. **Load Testing**: Test email service under load

### Monitoring

1. **Delivery Tracking**: Monitor email delivery success rates
2. **Error Tracking**: Track and alert on email failures
3. **Performance Metrics**: Monitor email sending performance
4. **API Limits**: Monitor ForwardEmail API usage limits

### Example Production Configuration

```python
# settings/production.py
EMAIL_BACKEND = 'apps.email_service.backends.ForwardEmailBackend'
FORWARD_EMAIL_BASE_URL = 'https://api.forwardemail.net'

# Use environment variables for sensitive data
import os
FORWARD_EMAIL_API_KEY = os.environ.get('FORWARD_EMAIL_API_KEY')
FORWARD_EMAIL_FROM = os.environ.get('FORWARD_EMAIL_FROM')

# Logging configuration
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'email_file': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': '/var/log/django/email.log',
        },
    },
    'loggers': {
        'apps.email_service': {
            'handlers': ['email_file'],
            'level': 'INFO',
            'propagate': True,
        },
    },
}
```

This documentation provides comprehensive coverage of the email service functionality, from basic setup to advanced usage patterns and troubleshooting.