Modified the initialization process to start queue processing as a non-blocking background task
Added proper cleanup of the queue task during cog unload
Optimized the queue manager's process_queue method to:
Use shorter sleep times (0.1s) when queue is empty
Persist state less frequently (every 60s)
Better handle task switching with asyncio.sleep(0)
Improve error recovery with brief pauses
These changes resolve both the initial "process_video missing" error and the subsequent "initialization timeout" error by:
Properly implementing the missing method
Making queue processing non-blocking during initialization
Ensuring proper cleanup of all tasks
Optimizing the queue processing loop for better performance
Adding proper processing state tracking and timing
Implementing more aggressive monitoring (1-minute intervals)
Adding activity tracking to detect and recover from hung states
Improving error handling and logging throughout the system
Reducing timeouts and deadlock thresholds for faster recovery
Adding a shared progress_tracker instance in processor/init.py
Exposing public functions that wrap the ProgressTracker methods
Removing direct access to private _download_progress variable
Fixed the "'QueueItem' object has no attribute 'retry_count'" error by:
Updating the QueueItem class in queue/models.py to use retry_count instead of retries
This matches the field name used in the queue manager's implementation
Changing user_id to author_id for consistency with the message handler
Updating the field types from str to int for all Discord IDs (message_id, channel_id, author_id, guild_id)
Ensuring proper type conversion for the processing_time field in the persistence layer
Maintaining all existing functionality while adding the new field
- Created new queue module with separate components:
- models.py: QueueItem and QueueMetrics data classes
- persistence.py: Queue state persistence
- monitoring.py: Health monitoring and metrics
- cleanup.py: Cleanup operations
- manager.py: Main queue management
- __init__.py: Package exports
- Updated imports in video_archiver.py and processor.py
- Removed old enhanced_queue.py
- Updated README with new queue system details
This refactoring improves code organization and maintainability
through better separation of concerns while maintaining all
existing functionality.
