Commit Graph

21 Commits

Author SHA1 Message Date
pacnpal
90a3090379 Improve video download stability and progress tracking:
- Increase retries and retry delay
- Reduce concurrent downloads for better stability
- Add exponential backoff with jitter
- Improve progress logging and error handling
- Add file size limits and timeouts
- Optimize chunk size and socket timeout
2024-11-15 14:44:21 +00:00
pacnpal
23fac11cfd Initial state: 📹 (video camera) when video is queued
Queue position: 1️⃣-5️⃣ showing position in queue
Download progress: 0️⃣2️⃣4️⃣6️⃣8️⃣🔟 showing download percentage (0%, 20%, 40%, 60%, 80%, 100%)
Processing state: ⚙️ when video is being processed
Compression progress: 🟨🟩 showing FFmpeg progress (0%, 50%, 100%)
Final state:  for success or  for failure
All status changes are now logged with message IDs for better tracking:

Queue position updates
Download progress at 20% intervals
Processing state changes
Compression progress
Success/failure states
Error conditions with detailed messages
2024-11-15 14:27:20 +00:00
pacnpal
f8d383a55a 📥 Pending: Shows items waiting to be processed
⚙️ Processing: Shows items currently being processed
 Completed: Shows successfully processed items
 Failed: Shows failed items
2024-11-15 14:12:43 +00:00
pacnpal
0d7b1d449f Made secure_delete_file and cleanup_downloads async in file_ops.py
Added chunked file processing and size limits to prevent blocking
Updated video_downloader.py to properly await secure_delete_file
Updated video_archiver.py to properly await cleanup_downloads
Simplified the secure deletion process while maintaining security
Added proper error handling and logging throughout
2024-11-15 13:47:28 +00:00
pacnpal
b4479c951b Fixed Discord Integration:
Added missing discord import
Added proper error handling for all discord operations
Improved error reporting for discord-specific failures
Enhanced Error Handling:
Added try/except blocks around all major operations
Implemented proper cleanup in finally blocks
Added more specific error messages for debugging
Queue Processing Improvements:
Ensured the queue continues processing even if individual items fail
Added better file cleanup to prevent resource leaks
Improved error reporting to help diagnose issues
Resource Management:
Added proper cleanup of downloaded files
Improved handling of missing discord resources
Better management of failed downloads
2024-11-15 13:36:26 +00:00
pacnpal
5e8f3c8b98 fix urls 2024-11-15 05:03:24 +00:00
pacnpal
4e8eb38ab7 Exception Hierarchy:
Added missing exceptions to utils/exceptions.py:
ResourceExhaustedError
ProcessingError
CleanupError
FileOperationError
Fixed exception imports in exceptions.py
Added proper re-exports of all exceptions
Maintained backward compatibility aliases
URL Processing:

Added pre-filtering for URLs before yt-dlp checks
Added common video platform patterns
Reduced error logging noise
Improved URL validation efficiency
Error Handling:

Added proper error types for all operations
Enhanced error messages with context
Added resource exhaustion checks
Improved error propagation
Code Organization:

Centralized exceptions in utils/exceptions.py
Proper re-exports in exceptions.py
Consistent error handling across components
Better error type hierarchy
2024-11-15 04:57:19 +00:00
pacnpal
074b6b246b Exception Handling:
Added DiscordAPIError to utils/exceptions.py
Fixed exception imports in config_manager.py
Ensured consistent exception hierarchy
Improved error messages and context
URL Processing:

Added pre-filtering for URLs before yt-dlp checks
Added common video platform patterns
Reduced error logging noise
Improved URL validation efficiency
FFmpeg Management:

Enhanced FFmpeg binary verification
Added robust subprocess handling
Improved cleanup of failed operations
Added detailed logging
Configuration:

Fixed ConfigurationError import
Improved error handling in config operations
Enhanced channel validation
Better error propagation
2024-11-15 04:54:35 +00:00
pacnpal
e4429a9d9e Fixed Exception Structure:
Added FileCleanupError to utils/exceptions.py
Created root exceptions.py for better organization
Fixed circular imports in utils/init.py
Updated imports in video_archiver.py and update_checker.py
Fixed FFmpeg Management:

Updated extraction logic for BtbN's new archive structure
Added fallback for backward compatibility
Better binary verification and permissions handling
Improved Error Handling:

Proper exception hierarchy
Better error propagation
More detailed error messages
Enhanced cleanup on errors
2024-11-15 04:34:35 +00:00
pacnpal
767f1140d1 Fixed FFmpeg Management:
Updated extraction logic to handle BtbN's new archive structure with binaries in bin directory
Added fallback to old structure for backward compatibility
Improved binary verification and permissions handling
Enhanced error handling during extraction and verification
Fixed Exception Handling:

Added missing FileCleanupError exception
Updated file operations to use proper exceptions
Improved error propagation and logging
Better cleanup error handling
Improved Resource Management:

Better cleanup of failed downloads
Proper handling of temporary files
Enhanced queue management
Improved file deletion with retries
Enhanced Error Reporting:

More detailed error messages
Better logging of cleanup operations
Proper error propagation through the system
Improved user feedback
The system now:

Properly handles FFmpeg's new archive structure
Better manages file operations and cleanup
Provides more detailed error messages
Cleans up resources properly
Shows appropriate error messages for different failure types
2024-11-15 04:30:03 +00:00
pacnpal
c144fb35ba The video downloading issues have been resolved by implementing comprehensive error handling and resource management:
FFmpeg is now properly managed:

Binaries are downloaded and verified on startup
Permissions are properly set
Hardware acceleration is detected and used when available
Resources are cleaned up properly
Error handling has been improved:

Specific exception types for different errors
Better error messages and logging
Appropriate reaction indicators
Enhanced component error handling
Resource management has been enhanced:

Failed downloads are tracked and cleaned up
Temporary files are handled properly
Queue management is more robust
Concurrent downloads are better managed
Verification has been strengthened:

FFmpeg binaries are verified
Video files are validated
Compression results are checked
Component initialization is verified
2024-11-15 04:18:57 +00:00
pacnpal
02966f1a66 Fixed FFmpeg/FFprobe Integration:
Added proper ffprobe binary download alongside FFmpeg
Added proper path handling for both binaries
Added verification of both binaries
Fixed FFmpeg compression by using subprocess directly instead of ffmpeg-python
Improved URL Detection:

Switched from regex patterns to yt-dlp simulation for URL detection
Added better error handling for URL checking
Added detailed logging of URL detection results
Reduced Logging Noise:

Only log when a message contains a video URL
Only log URL check errors for actual URLs (containing http://, https://, or www.)
Removed unnecessary debug logging
2024-11-15 03:54:42 +00:00
pacnpal
46af1a31b7 Enhanced FFmpeg Integration:
Added robust error handling and logging
Improved binary verification and initialization
Added proper GPU detection and hardware acceleration
Optimized encoding parameters for different content types
Improved File Operations:

Added retry mechanisms for file operations
Enhanced temporary directory management
Improved cleanup of failed downloads
Added proper permission handling
Enhanced Queue Management:

Fixed queue manager initialization
Added better error recovery
Improved status tracking and logging
Enhanced cleanup of failed items
Better Error Handling:

Added comprehensive exception hierarchy
Improved error logging and reporting
Added fallback mechanisms for failures
Enhanced error recovery strategies
2024-11-15 03:48:56 +00:00
pacnpal
3e50faec75 Switching from regex patterns to yt-dlp simulation for URL detection:
Removed the regex-based URL detection
Added proper yt-dlp simulation to check if URLs are supported
Added better error handling for URL checking
Added detailed logging of URL detection results
Improving FFmpeg integration:

Added proper ffprobe binary download alongside FFmpeg
Added better verification of both binaries
Added retry mechanisms for binary downloads
Added proper cleanup of failed downloads
Enhancing error handling and logging:

Added detailed logging throughout the system
Added better error recovery mechanisms
Added proper cleanup of temporary files
Added better tracking of failed operations
2024-11-15 03:40:53 +00:00
pacnpal
8503fc6fdd Enhanced FFmpeg Integration:
Added robust error handling and logging
Improved binary verification and initialization
Added proper GPU detection and hardware acceleration
Optimized encoding parameters for different content types
Improved File Operations:

Added retry mechanisms for file operations
Enhanced temporary directory management
Improved cleanup of failed downloads
Added proper permission handling
Enhanced Queue Management:

Fixed queue manager initialization
Added better error recovery
Improved status tracking and logging
Enhanced cleanup of failed items
Better Error Handling:

Added comprehensive exception hierarchy
Improved error logging and reporting
Added fallback mechanisms for failures
Enhanced error recovery strategies
2024-11-15 03:21:25 +00:00
pacnpal
0a46cee479 fix: Change URL validation to use regex patterns
- Replaced extractor.suitable() with direct regex pattern matching
- Store both pattern and site name for better debugging
- Added debug logging for URL matching
- Improved error handling
2024-11-15 01:59:03 +00:00
pacnpal
63664e3c94 fix: URL extraction error in is_supported_url method
- Fixed 'str' object has no attribute 'suitable' error
- Now properly instantiating extractors before calling suitable()
- Improved error handling in URL validation
2024-11-15 01:50:19 +00:00
pacnpal
8822c85b6f Add context menu command for birthday role assignment. Users with allowed roles can now give birthday roles via right-click menu. 2024-11-15 01:36:19 +00:00
pacnpal
da9a6e40f9 fix: Properly validate URLs using extractors' suitable() method 2024-11-15 00:53:56 +00:00
pacnpal
f3d28231e8 fix imports 2024-11-14 23:45:36 +00:00
pacnpal
d8ca4e5b8e fix imports 2024-11-14 23:24:48 +00:00