ffmpeg从MP4中取出wav音频

### 创建用于Wav2Lip训练的数据集 #### 数据收集 构建高质量的音视频同步数据集对于提升Wav2Lip模型的表现至关重要。理想情况下，应获取大量带有清晰面部特写镜头且音频与口型完美匹配的真实场景视频片段[^4]。 #### 数据预处理工具准备 为了简化工作流程并确保一致性，在开始前需安装必要的依赖库以及克隆官方仓库中的辅助脚本： ```bash pip install ffmpeg-python moviepy opencv-python-headless numpy pandas tqdm git clone --depth=1 https://github.com/Rudrabha/Wav2Lip.git cd Wav2Lip ``` #### 提取帧序列 利用`ffmpeg`命令行工具可以从原始视频材料中提取连续静止画面作为后续分析的基础素材。建议设定固定间隔时间戳来抽取样本帧，从而减少冗余信息量的同时保持足够的时空分辨率。 ```python import os from pathlib import Path import subprocess as sp def extract_frames(video_path, output_dir, fps=25): """ Extract frames from video at specified FPS rate. Args: video_path (str): Input video file path. output_dir (str): Directory to save extracted images into. fps (int): Frames per second extraction frequency. Returns: None """ if not isinstance(fps, int) or fps <= 0: raise ValueError('Invalid frame rate value') cmd = [ 'ffmpeg', '-i', str(Path(video_path).absolute()), '-vf', f'fps={fps}', '{}/%06d.png'.format(output_dir), '-hide_banner' ] try: process = sp.run(cmd, check=True, stdout=sp.PIPE, stderr=sp.STDOUT) except Exception as e: print(process.stdout.decode()) raise RuntimeError(e) if __name__ == '__main__': input_video = './example.mp4' target_folder = './frames_output/' os.makedirs(target_folder, exist_ok=True) extract_frames(input_video, target_folder) ``` #### 裁剪人脸区域 针对每一张抓取出的画面应用MTCNN等人脸检测算法定位嘴巴位置，并据此截取感兴趣子图以便于聚焦局部特征学习过程。此步骤有助于加速计算效率并且改善最终效果。 ```python from mtcnn.mtcnn import MTCNN import cv2 detector = MTCNN() for img_file in sorted(os.listdir('./frames_output')): image = cv2.imread(os.path.join('./frames_output/', img_file)) result = detector.detect_faces(image) bounding_boxes = [] for face in result: box = face['box'] keypoints = face['keypoints'] mouth_left_x = min(keypoints['mouth_left'][0], keypoints['mouth_right'][0]) mouth_right_x = max(keypoints['mouth_left'][0], keypoints['mouth_right'][0]) mouth_top_y = min(keypoints['mouth_top'][1], keypoints['bottom_lip_bottom'][1]) mouth_bot_y = max(keypoints['top_lip_top'][1], keypoints['bottom_lip_bottom'][1]) margin_w = abs(mouth_right_x - mouth_left_x) * .3 margin_h = abs(mouth_bot_y - mouth_top_y) * .7 cropped_image = image[ int(max(0, round(box[1]-margin_h))): int(min(round(box[1]+box[3]+margin_h), image.shape[0])), int(max(0, round(box[0]-margin_w))): int(min(round(box[0]+box[2]+margin_w), image.shape[1])) ] resized_cropped_img = cv2.resize(cropped_image, dsize=(96, 96)) out_filename = '{}_cropped.jpg'.format(img_file.split('.')[0]) cv2.imwrite(out_filename, resized_cropped_img) ``` #### 同步校验 最后一步是对齐后的视听对进行严格的质量审查，剔除那些存在明显偏差或者不符合标准规格的例子。这可能涉及到人工审核环节或是借助自动化手段完成初步筛选作业。 通过上述操作可以获得适配Wav2Lip框架需求的标准格式化数据集合，进而支持更深入的研究探索活动。