Commit Graph

28 Commits

Author SHA1 Message Date
pacnpal
a4ca6e8ea6 Core Systems:
Component-based architecture with lifecycle management
Enhanced error handling and recovery mechanisms
Comprehensive state management and tracking
Event-driven architecture with monitoring
Queue Management:

Multiple processing strategies for different scenarios
Advanced state management with recovery
Comprehensive metrics and health monitoring
Sophisticated cleanup system with multiple strategies
Processing Pipeline:

Enhanced message handling with validation
Improved URL extraction and processing
Better queue management and monitoring
Advanced cleanup mechanisms
Overall Benefits:

Better code organization and maintainability
Improved error handling and recovery
Enhanced monitoring and reporting
More robust and reliable system
2024-11-16 05:01:29 +00:00
pacnpal
e66af6e844 Added timeout handling and force cleanup in VideoArchiver cog
Added proper cancellation and requeuing of downloads in VideoProcessor
Added cancellable logger and process cleanup in VideoDownloader
Added shutdown flag and force_stop capability in EnhancedQueueManager
Added process tracking and kill_all_processes method in FFmpegManager
The changes ensure that:

Active downloads are paused and requeued when unload is called
If cleanup takes too long, force cleanup kicks in
All resources are properly cleaned up, even in case of timeout
Downloads can be safely cancelled and resumed later
No processes are left hanging during unload
2024-11-15 15:48:03 +00:00
pacnpal
b4479c951b Fixed Discord Integration:
Added missing discord import
Added proper error handling for all discord operations
Improved error reporting for discord-specific failures
Enhanced Error Handling:
Added try/except blocks around all major operations
Implemented proper cleanup in finally blocks
Added more specific error messages for debugging
Queue Processing Improvements:
Ensured the queue continues processing even if individual items fail
Added better file cleanup to prevent resource leaks
Improved error reporting to help diagnose issues
Resource Management:
Added proper cleanup of downloaded files
Improved handling of missing discord resources
Better management of failed downloads
2024-11-15 13:36:26 +00:00
pacnpal
887547473c Added proper physical GPU detection on Linux/Windows/macOS
Separated physical GPU detection from FFmpeg encoder support
Only enable GPU encoding when both physical GPU and FFmpeg support exist
Added multiple detection methods for better reliability
2024-11-15 05:26:10 +00:00
pacnpal
8e581d660e Fixed NVENC preset compatibility
Added proper NVENC preset validation
Separated CPU and GPU presets
Added preset conversion logic for GPU encoding
2024-11-15 05:22:38 +00:00
pacnpal
f8fe409cdd Architecture Changes:
Moved FFmpeg manager to cog level instead of per-guild
Created single shared FFmpeg manager instance in VideoArchiver class
Passed shared FFmpeg manager to VideoProcessor and VideoDownloader
Removed redundant FFmpeg downloads for each guild
Component Management:

Updated VideoArchiver to initialize one FFmpeg manager
Modified guild components to remove FFmpeg manager
Updated component cleanup to handle shared resources
Improved resource initialization order
Resource Efficiency:

Eliminated duplicate FFmpeg binary downloads
Reduced disk space usage
Improved initialization time
Better resource sharing across guilds
Error Handling:

Added proper cleanup for shared resources
Improved error propagation
Enhanced initialization error handling
Better component lifecycle management
2024-11-15 05:17:21 +00:00
pacnpal
06fcf55d90 Encoding Parameters:
Added minimum video bitrate (500 Kbps) to prevent quality degradation
Fixed audio bitrate calculation with proper min/max per channel
Added standard audio sample rate (48 kHz)
Improved bitrate allocation between video and audio
2024-11-15 05:10:59 +00:00
pacnpal
2d92a3dca0 FFmpeg Binary Management:
Fixed tar.xz extraction by separating decompression and extraction steps
Added proper error handling for archive operations
Improved binary verification process
Enhanced cleanup of temporary files
Exception Handling:

Added missing exceptions to utils/exceptions.py
Fixed exception imports in exceptions.py
Added proper error types for all operations
Improved error messages and context
URL Processing:

Added pre-filtering for URLs before yt-dlp checks
Added common video platform patterns
Reduced error logging noise
Improved URL validation efficiency
Code Organization:

Centralized exceptions in utils/exceptions.py
Proper re-exports in exceptions.py
Consistent error handling across components
Better file operation handling
2024-11-15 05:00:47 +00:00
pacnpal
7e9e193178 URL Processing:
Added URL pre-filtering to avoid unnecessary yt-dlp checks
Added common video platform patterns for quick filtering
Reduced error logging noise from non-URL words
Improved URL validation efficiency
FFmpeg Management:

Enhanced FFmpeg binary verification
Added robust error handling for subprocess calls
Improved cleanup of failed operations
Added detailed logging for binary operations
Error Handling:

Fixed exception hierarchy in utils/exceptions.py
Added proper error types for different failure scenarios
Enhanced error messages with more context
Improved error propagation through the system
Process Flow:

Added proper timeout handling for subprocess calls
Enhanced environment variable handling
Better cleanup after failures
Added retry mechanisms for failed operations
2024-11-15 04:51:17 +00:00
pacnpal
9824469984 FFmpeg Management:
Updated extraction logic to handle BtbN's new archive structure with binaries in the bin directory
Added fallback to old structure for backward compatibility
Improved binary verification and permissions handling
Enhanced error handling during extraction and verification
Error Handling:

Added specific exception types for different errors
Better error propagation through the system
Improved error logging and user feedback
Enhanced cleanup on errors
Resource Management:

Better cleanup of failed downloads
Proper handling of temporary files
Enhanced queue management
Improved file deletion with retries
Verification:

FFmpeg binary verification with version checks
Video file integrity verification
Compression result verification
Archive structure verification
The system now:

Properly handles FFmpeg's new archive structure
Better manages binary downloads and verification
Provides more detailed error messages
Cleans up resources properly
Shows appropriate reactions for different error types
2024-11-15 04:20:17 +00:00
pacnpal
c144fb35ba The video downloading issues have been resolved by implementing comprehensive error handling and resource management:
FFmpeg is now properly managed:

Binaries are downloaded and verified on startup
Permissions are properly set
Hardware acceleration is detected and used when available
Resources are cleaned up properly
Error handling has been improved:

Specific exception types for different errors
Better error messages and logging
Appropriate reaction indicators
Enhanced component error handling
Resource management has been enhanced:

Failed downloads are tracked and cleaned up
Temporary files are handled properly
Queue management is more robust
Concurrent downloads are better managed
Verification has been strengthened:

FFmpeg binaries are verified
Video files are validated
Compression results are checked
Component initialization is verified
2024-11-15 04:18:57 +00:00
pacnpal
46af1a31b7 Enhanced FFmpeg Integration:
Added robust error handling and logging
Improved binary verification and initialization
Added proper GPU detection and hardware acceleration
Optimized encoding parameters for different content types
Improved File Operations:

Added retry mechanisms for file operations
Enhanced temporary directory management
Improved cleanup of failed downloads
Added proper permission handling
Enhanced Queue Management:

Fixed queue manager initialization
Added better error recovery
Improved status tracking and logging
Enhanced cleanup of failed items
Better Error Handling:

Added comprehensive exception hierarchy
Improved error logging and reporting
Added fallback mechanisms for failures
Enhanced error recovery strategies
2024-11-15 03:48:56 +00:00
pacnpal
3e50faec75 Switching from regex patterns to yt-dlp simulation for URL detection:
Removed the regex-based URL detection
Added proper yt-dlp simulation to check if URLs are supported
Added better error handling for URL checking
Added detailed logging of URL detection results
Improving FFmpeg integration:

Added proper ffprobe binary download alongside FFmpeg
Added better verification of both binaries
Added retry mechanisms for binary downloads
Added proper cleanup of failed downloads
Enhancing error handling and logging:

Added detailed logging throughout the system
Added better error recovery mechanisms
Added proper cleanup of temporary files
Added better tracking of failed operations
2024-11-15 03:40:53 +00:00
pacnpal
8503fc6fdd Enhanced FFmpeg Integration:
Added robust error handling and logging
Improved binary verification and initialization
Added proper GPU detection and hardware acceleration
Optimized encoding parameters for different content types
Improved File Operations:

Added retry mechanisms for file operations
Enhanced temporary directory management
Improved cleanup of failed downloads
Added proper permission handling
Enhanced Queue Management:

Fixed queue manager initialization
Added better error recovery
Improved status tracking and logging
Enhanced cleanup of failed items
Better Error Handling:

Added comprehensive exception hierarchy
Improved error logging and reporting
Added fallback mechanisms for failures
Enhanced error recovery strategies
2024-11-15 03:21:25 +00:00
pacnpal
826f5e1b3e Improved FFmpeg downloader error handling and permissions 2024-11-15 01:31:37 +00:00
pacnpal
190e689919 Changed FFmpeg binary download location from /tmp/ffmpeg to videoarchiver/bin for better organization and persistence 2024-11-15 01:26:58 +00:00
pacnpal
f3d28231e8 fix imports 2024-11-14 23:45:36 +00:00
pacnpal
e094dbfdb6 fix imports 2024-11-14 23:12:21 +00:00
pacnpal
171453ce57 fix imports 2024-11-14 23:09:28 +00:00
pacnpal
0533deaca3 fix imports 2024-11-14 23:07:04 +00:00
pacnpal
dfba1a4e30 fix imports 2024-11-14 23:05:16 +00:00
pacnpal
2ba89edbd9 refactor: Improve ffmpeg_downloader module
- Enhanced Docker compatibility with proper permissions
- Better error handling and logging
- Improved download and extraction process
- Added detailed type hints and documentation
2024-11-14 22:37:42 +00:00
pacnpal
2f449e6120 refactor: Clean up encoder_params module
- Improved parameter organization
- Better error handling and logging
- More consistent code style
- Added detailed comments for parameter choices
2024-11-14 22:36:50 +00:00
pacnpal
4719f567c6 refactor: Update video_analyzer to use ffprobe directly
- Removed dependency on ffmpeg-python package
- Added direct ffprobe command execution
- Improved error handling and logging
- Better organization of analysis methods
2024-11-14 22:36:01 +00:00
pacnpal
9beda06e34 fix: Update imports in ffmpeg_manager module
- Updated imports to use full package paths consistently
- Ensured all imports use videoarchiver.ffmpeg namespace
2024-11-14 22:33:47 +00:00
pacnpal
e5ea9f7bed fix: Update imports for Red-DiscordBot compatibility
- Modified imports to use full package paths
- Fixed circular import issues
- Updated imports to work within cog structure
2024-11-14 22:33:18 +00:00
pacnpal
c9872b123d fix 2024-11-14 22:31:37 +00:00
pacnpal
5454630591 refactor: Split FFmpeg manager into modular components
- Created ffmpeg package with specialized modules
- Improved Docker compatibility using /tmp
- Better permission handling
- More robust error handling
- Separated concerns for easier maintenance
- Simplified imports through __init__.py
2024-11-14 22:26:32 +00:00