Properly exposing QueueProcessor in processor/init.py
Updating queue/init.py to import and expose all necessary types (QueuePriority, ProcessingMetrics)
Ensuring consistent import patterns across the codebase
The changes maintain the cog's architecture while fixing the import issues:
Module reloading system now includes all dependencies
Package hierarchy properly exposes all needed types
Import structure is consistent across components
All necessary types and models are accessible
core/types.py - Contains shared interfaces and types
queue/types.py - Contains queue-specific types
Components now depend on interfaces rather than concrete implementations
Broke cyclic dependencies:
Removed direct imports between queue_processor.py and manager.py
Removed circular dependencies between core and processor modules
Components now communicate through well-defined interfaces
Improved architecture:
Clear separation of concerns
Better dependency management
More maintainable and testable code
Proper use of dependency injection
Updating queue_processor.py to use the shared types
Updating manager.py to use the correct imports and shared types
The cyclic dependency has been resolved by:
Moving shared types to a separate module
Having queue_processor.py only import from shared modules
Having manager.py use the QueueProcessor from queue/processor.py
Updated base.py to use absolute imports
Updated cleanup.py to use absolute imports
Updated events.py to use absolute imports
Updated error_handler.py to use absolute imports
Updated guild.py to use absolute imports
Updated initialization.py to use absolute imports
Updated lifecycle.py to use absolute imports
Updated response_handler.py to use absolute imports
Updated settings.py to use absolute imports
In the core/commands module:
Updated archiver_commands.py to use absolute imports
Updated database_commands.py to use absolute imports
Updated settings_commands.py to use absolute imports
Left init.py unchanged as its relative imports are appropriate
In the processor module:
Updated core.py to use absolute imports
Updated processor/init.py to use absolute imports
Updated queue_handler.py to use absolute imports
Updated queue_processor.py to use absolute imports
Updated status_display.py to use absolute imports
Updated cleanup_manager.py to use absolute imports
from videoarchiver.utils import progress_tracker
Updated the main init.py to properly handle module reloading:
Added 'videoarchiver.processor' and 'videoarchiver.processor.core' to modules_to_reload
Added explicit import and reload of the processor module
These changes ensure that:
The progress_tracker is properly imported from the utils package
All necessary modules are reloaded during initialization
Circular imports are avoided
The package structure maintains proper dependency flow
Implemented proper singleton pattern in ProgressTracker class
Created shared instance in utils/init.py
Updated all files to use relative imports correctly
Removed duplicate progress_tracker.py from processor/
Fixed circular imports by removing ProgressTracker from processor.py re-exports
Updated all files to import REACTIONS from constants.py instead of reactions.py:
queue_processor.py
message_handler.py
reactions.py (now only contains reaction handling functions)
events.py (now imports REACTIONS from constants.py and handle_archived_reaction from reactions.py)
core.py
queue_handler.py
processor/init.py
Added process_queue method to EnhancedVideoQueueManager and updated its initialization
Updated component_manager.py to use EnhancedVideoQueueManager correctly
Fixed circular imports in the core module by:
Moving initialization logic to lifecycle.py
Making initialization.py provide thin wrappers that delegate to lifecycle.py
Ensuring proper import order in base.py
Verified all module init.py files are properly exposing their components:
core/init.py exposes VideoArchiver
queue/init.py exposes EnhancedVideoQueueManager and dependencies
processor/init.py exposes VideoProcessor and related components
commands/init.py exposes command setup functions
The import chain is now clean:
base.py imports from lifecycle.py
lifecycle.py contains all initialization logic
initialization.py delegates to lifecycle.py
No circular dependencies
All components are properly exposed through their respective init.py files
Component-based architecture with lifecycle management
Enhanced error handling and recovery mechanisms
Comprehensive state management and tracking
Event-driven architecture with monitoring
Queue Management:
Multiple processing strategies for different scenarios
Advanced state management with recovery
Comprehensive metrics and health monitoring
Sophisticated cleanup system with multiple strategies
Processing Pipeline:
Enhanced message handling with validation
Improved URL extraction and processing
Better queue management and monitoring
Advanced cleanup mechanisms
Overall Benefits:
Better code organization and maintainability
Improved error handling and recovery
Enhanced monitoring and reporting
More robust and reliable system
New database functionality and commands
Additional video format support (MKV)
Message duration now in hours (0-168)
Enhanced error handling and logging
Queue persistence and management improvements
Hardware acceleration updates
More detailed command descriptions
Improved performance & limitations documentation
Method delegates to queue_handler.process_video
Maintains proper separation of concerns between components
Preserves the existing video processing logic
Adding proper processing state tracking and timing
Implementing more aggressive monitoring (1-minute intervals)
Adding activity tracking to detect and recover from hung states
Improving error handling and logging throughout the system
Reducing timeouts and deadlock thresholds for faster recovery
base.py: Main cog class and initialization logic
cleanup.py: Resource cleanup functionality
commands.py: Command handlers
events.py: Event listeners
guild.py: Guild component management
init.py: Module exports
Improved code organization by:
Separating concerns into focused modules
Maintaining clear dependencies between modules
Keeping related functionality together
Making the codebase more maintainable
Preserved all existing functionality while making the code more modular and easier to maintain.
Properly storing and handling the db instance
Ensuring the db is passed to the queue_handler when available
The QueueHandler class was already properly set up to handle the db parameter, so no changes were needed there. The database integration should now work correctly for:
Checking if videos are already archived
Adding newly archived videos to the database
Providing archived video information to users
Adding a shared progress_tracker instance in processor/init.py
Exposing public functions that wrap the ProgressTracker methods
Removing direct access to private _download_progress variable
Fixed the "'QueueItem' object has no attribute 'retry_count'" error by:
Updating the QueueItem class in queue/models.py to use retry_count instead of retries
This matches the field name used in the queue manager's implementation
Core processing logic in VideoProcessor class
Dedicated message handling in MessageHandler
Queue operations in QueueHandler
Progress tracking in ProgressTracker
Reaction management in reactions.py
Improved Maintainability:
Each component has a single responsibility
Easier to test individual components
Better code organization and readability
Reduced file sizes for better version control
Better Resource Management:
Centralized progress tracking
Isolated queue operations
Cleaner cleanup processes
Optimized Imports:
Components can be imported individually as needed
Main processor.py provides backward compatibility
Clear module interface through init.py
