Update VideoArchiver README with detailed queue system architecture

- Added detailed description of modular queue components:
  - Models for data structures
  - Persistence for state management
  - Monitoring for health checks
  - Cleanup for resource management
  - Manager for core operations

- Expanded metrics and monitoring section
- Added state management details
- Improved feature descriptions for new queue architecture
This commit is contained in:
pacnpal
2024-11-15 18:19:05 +00:00
parent b1eafbb01d
commit 6d1e373c43

View File

@@ -11,13 +11,16 @@ A powerful video archiving cog that automatically downloads and reposts videos f
- Configurable video quality and format - Configurable video quality and format
- Automatic file size optimization for Discord limits - Automatic file size optimization for Discord limits
- **Enhanced Queue System** - **Modular Queue System**
- Priority-based processing - Priority-based processing with state persistence
- Queue persistence across bot restarts - Efficient resource management and monitoring
- Performance metrics tracking - Real-time performance metrics and health checks
- Automatic cleanup and memory management - Automatic cleanup and memory optimization
- Real-time queue status monitoring - Component-based architecture:
- Detailed performance analytics - Queue state persistence and recovery
- Health monitoring and metrics tracking
- Resource cleanup and maintenance
- Core queue operations management
- **Channel Management** - **Channel Management**
- Flexible channel monitoring (specific channels or all) - Flexible channel monitoring (specific channels or all)
@@ -80,21 +83,32 @@ All commands support both prefix and slash command syntax:
- **`vas_enable [sites...]`**: Enable specific sites (empty for all) - **`vas_enable [sites...]`**: Enable specific sites (empty for all)
- **`vas_list`**: List available and enabled sites - **`vas_list`**: List available and enabled sites
## Queue System ## Queue System Architecture
The enhanced queue system provides: The queue system is built with a modular, component-based architecture:
### Basic Metrics ### Core Components
- Pending/Processing/Completed/Failed counts - **Models**: Data structures for queue items and metrics
- Success rate percentage - **Persistence**: Queue state persistence and recovery
- Average processing time - **Monitoring**: Health checks and performance metrics
- **Cleanup**: Resource management and maintenance
- **Manager**: Core queue operations and coordination
### Detailed Metrics ### Metrics and Monitoring
- Total processed videos - Real-time queue status and health monitoring
- Total failures - Comprehensive performance metrics:
- Peak memory usage - Processing counts and success rates
- Last cleanup time - Hardware acceleration statistics
- Real-time queue state - Memory usage tracking
- Error distribution analysis
- Automatic recovery from failures
- Resource usage optimization
### State Management
- Persistent queue state across bot restarts
- Efficient memory management
- Automatic cleanup of old entries
- Priority-based processing
## Message Templates ## Message Templates