markov-discord/cline_docs/systemPatterns.md

System Patterns

Last Updated: 2024-12-27

High-level Architecture

Current System

Discord Events -> Message Processing -> SQLite Storage
                                   -> Markov Generation

Proposed LLM Integration

Discord Events -> Message Processing -> SQLite Storage
                                   -> Response Generator
                                      ├─ Markov Chain
                                      ├─ LLM
                                      └─ Response Selector

Core Technical Patterns

Data Storage

• SQLite database using TypeORM
• Entity structure:
  • Guild (server)
  • Channel (per-server channels)
  • Messages (training data)

Message Processing

1. Current Flow:

   • Message received
   • Filtered for human authorship
   • Stored in database with metadata
   • Used for Markov chain training

2. Enhanced Flow:

   • Add message embedding generation
   • Store context window
   • Track conversation threads

Response Generation

Current (Markov)

interface MarkovGenerateOptions {
  filter: (result) => boolean;
  maxTries: number;
  startSeed?: string;
}

Proposed (Hybrid)

interface ResponseGenerateOptions {
  contextWindow: Message[];
  temperature: number;
  maxTokens: number;
  startSeed?: string;
  forceProvider?: 'markov' | 'llm' | 'hybrid';
}

Data Flow

Training Pipeline

1. Message Collection

   • Discord channel history
   • JSON imports
   • Real-time messages

2. Processing

   • Text cleaning
   • Metadata extraction
   • Embedding generation

3. Storage

   • Raw messages
   • Processed embeddings
   • Context relationships

Response Pipeline

1. Context Gathering

   • Recent messages
   • Channel history
   • User interaction history

2. Generation Strategy

   • Short responses: Markov chain
   • Complex responses: LLM
   • Hybrid: LLM-guided Markov chain

3. Post-processing

   • Response filtering
   • Token limit enforcement
   • Attachment handling

Key Technical Decisions

LLM Integration

1. Local Embedding Model

   • Use sentence-transformers for message embedding
   • Store embeddings in SQLite
   • Enable semantic search

2. Response Generation

   • Primary: Use OpenAI API
   • Fallback: Use local LLM
   • Hybrid: Combine with Markov output

3. Context Management

   • Rolling window of recent messages
   • Semantic clustering of related content
   • Thread-aware context tracking

Performance Requirements

1. Response Time

   • Markov: < 100ms
   • LLM: < 2000ms
   • Hybrid: < 2500ms

2. Memory Usage

   • Max 1GB per guild
   • Batch processing for large imports
   • Regular cleanup of old contexts

3. Rate Limiting

   • Discord API compliance
   • LLM API quota management
   • Fallback mechanisms