This document describes the video capture and processing infrastructure in Deep-Live-Cam. It covers the systems responsible for extracting frames from video files, capturing frames from live camera feeds, assembling processed frames back into videos, and managing temporary file storage during processing operations.
This page focuses on the low-level video I/O operations. For information about how frames are processed after capture, see Processing Pipeline Architecture. For details about face detection within captured frames, see Face Detection and Analysis.
Sources: modules/utilities.py1-210 modules/capturer.py1-33 modules/video_capture.py1-160
Deep-Live-Cam employs a multi-stage pipeline for video processing that involves frame extraction, parallel processing, video reconstruction, and audio restoration. The pipeline differs significantly between file-based video processing and live camera capture.
Sources: modules/utilities.py64-119 modules/utilities.py148-168
Sources: modules/video_capture.py37-128 modules/video_capture.py135-156
Deep-Live-Cam relies heavily on FFmpeg for video operations. All FFmpeg commands are executed through the run_ffmpeg() utility function with standardized parameters.
The run_ffmpeg() function in modules/utilities.py19-39 provides a standardized wrapper for all FFmpeg operations:
Key Features:
-hwaccel auto and -hwaccel_output_format auto enable automatic hardware acceleration when available modules/utilities.py24-25-threads is set via modules.globals.execution_threads modules/utilities.py26Sources: modules/utilities.py19-39
The frame extraction system converts video files into individual PNG images stored in a temporary directory structure.
The extract_frames() function at modules/utilities.py64-77 executes the extraction:
Key Parameters:
-vf format=rgb24 uses a video filter for faster RGB conversion modules/utilities.py73-vsync 0 prevents frame duplication modules/utilities.py74%04d.png creates sequentially numbered files modules/utilities.py75Sources: modules/utilities.py64-77
The modules/capturer.py module provides functions for capturing individual frames from video files using OpenCV's VideoCapture API.
The get_video_frame() function at modules/capturer.py7-26 captures a specific frame from a video file:
Key Features:
CAP_PROP_FOURCC to MJPEG for correct color space handling modules/capturer.py11modules.globals.color_correction is enabled modules/capturer.py14-15gpu_cvt_color for BGR to RGB conversion when color correction is active modules/capturer.py23Sources: modules/capturer.py7-26 modules/predicter.py15-28
The VideoCapturer class in modules/video_capture.py manages live camera capture with platform-specific optimizations.
The start() method at modules/video_capture.py37-107 implements platform-specific initialization:
Windows Capture Strategy:
On Windows, the system prioritizes CAP_DSHOW (DirectShow) to preserve device mapping from pygrabber, with CAP_MSMF and CAP_ANY as fallbacks modules/video_capture.py40-65 It negotiates MJPG at construction to avoid bandwidth-limited YUYV modules/video_capture.py48-54
FPS Measurement:
Because CAP_PROP_FPS is often unreliable on DirectShow, the system uses _measure_fps() to empirically determine the frame rate by timing a burst of frames modules/video_capture.py98-100 modules/video_capture.py135-156
Sources: modules/video_capture.py37-107 modules/video_capture.py135-156 modules/platform_info.py45-58
The create_video() function at modules/utilities.py80-190 implements a sophisticated hardware-accelerated encoding pipeline.
The system detects available execution providers to select the optimal hardware encoder:
| Hardware | Encoder (H.264) | Encoder (H.265) | Logic Source |
|---|---|---|---|
| NVIDIA (CUDA) | h264_nvenc | hevc_nvenc | modules/utilities.py92-103 |
| AMD/Intel (DML) | h264_amf | hevc_amf | modules/utilities.py113-123 |
| CPU | libx264 | libx265 | modules/utilities.py132-138 |
Fallback Mechanism:
If hardware encoding fails, the system automatically falls back to software encoding (libx264 or libx265) with medium presets modules/utilities.py173-189
Sources: modules/utilities.py80-190
Deep-Live-Cam handles file paths using standard Python os and pathlib modules, ensuring compatibility across different operating systems.
| Function | Purpose | Implementation |
|---|---|---|
get_temp_directory_path() | Calculate temp directory path | modules/utilities.py126-130 |
get_temp_output_path() | Get path for intermediate video | modules/utilities.py133-135 |
get_temp_frame_paths() | Collect frame paths | Uses glob.escape for safe pattern matching modules/utilities.py121-123 |
Sources: modules/utilities.py121-135
Refresh this wiki