Video quality testing
4 likes3,044 views
The document outlines video quality measurement methods, which consist of three testing levels: media stream analyzers, sender side measurements, and receiver side evaluations. It details the roles and techniques for each type of testing, including IP-based assessment and re-synced video comparison for accurate quality analysis. Additionally, it lists recommended testing tools for various scenarios, emphasizing the importance of sender-side quality checks despite user focus on client-side video experience.
Video quality testing
- 1. Video and Video over IP Testing Methods There are three levels of testing which apply to video quality measurements: 1) Media Stream analyzers 2) Encoder output Quality measurements 3) Client side media quality measurements Media Analyzer The media stream analysis is done at the output of the sender’s Encoder or multiplexer output. The media Analyzer’s roles are: 1. Verify that the audio and video streams comply with the coding standard 2. Understand the encoding parameters including: GOP structure, encoder profile and level, resolution, fps, VBV, use of CABAC, UEP etc. 3. Enables tweaking parameters which effects encoder latency, stream robustness Sender’s side quality measurement The quality measurement on the sender’s side is based on full payload quality measurements techniques like PSNR and SSIM. Although the user perceives only the video quality on the client side, it’s also important to check the quality on the sender’s side. Sender’s side analysis roles include: 1) Improving video quality regardless of the network impairments. 2) Testing video quality at the encoder’s output as base level quality 3) Enabling tweaking of parameters for robustness/quality/latency tradeoffs Receiver side quality measurement There are two types of Receiver side testing 1) IP based testing 2) Re-synced Video comparison IP Based testing IP based testing uses the packet low and jitter data along with packet header information to assess the video quality and does not use video decoding and full payload comparison like transmitter side testing. The reasoning for using this method is that the source file is not available, there are not enough resources to do full payload analysis or the packet loss will cause miss-synchronization that would not enable frame-to-frame comparison and
- 2. will require re-synchronization. Re-Synced video comparison Re-synced video methods uses image code embedded on each frame understand the frame number and resync the destination video with the original video. Testing diagram The testing of the system is done according to this the following diagram: ReceiverTransmitter Video Encoder Audio Encoder Stream Analyzer Mux / Packetizer Video Quality testing Network Emulator IP Based video quality probe Testing Tools This section reviews testing tools for each of the testing scenarios described above Methods Tool Media Analyzers Interra H264 Analyzer (preferred) – good (recommended) Tektronics E4A - good Mindego H264 Analyzer - average CodecVisa - crash, can’t find headers DVBSnoop – free text based tool used by WiFiDi. H264 Analyzer – text based hard to work with. for scripting Video Quality MSU VQMT – free for SD Interra H264 Analyzer YUV tool – included in interra YUVTools – free for eval. Provides PSNR/SSIM Receiver side probes Interra – Orion – does not support Promiscuous mode Telchemy – VQCapture (recommended) Tektroniks VQS – does not support elementary streams only M2TS HW based solutions
- 3. Operating Testing Equipment Media Analyzers -Interra Interra – open the raw H.264 video file review if error exists, check Qp per frame graph average bitrate etc. PSNR Measurements – MSU VQMT Operation: 1. Select a compressed file with high quality 2. Extract the YUV of the file using VLC command: vlc <Source File Name> :sout=#transcode{venc=ffmpeg{keyint=1},vcodec=I420}:std{access=file,mux=ra w,dst=<Source File Name>} 3. Play and transmit the file while adding a "Sync Frame" in the form of a Black/White marker on the frame every two seconds. 4. Record the video output of the encoder on the source 5. Convert the video output to YUV using the above command 6. Use the MSU VQMT software to compare the files as described in the image below: 7. The results for PSNR are shown below, SSIM results looks usually the same.
- 4. 8. This diagram is for Quality/Frame. To compare with other videos/ configuration it is convertible to a candle stick diagram with min/max and mean quality Receiver Side IP Based testing - VQCapture The testing configuration of IP Based testing is charted below: Network EmulatorVideo/Audio over RTP ES VQCapture File Analysis The packet recording is saved by wireshark on the PC after the network emulator recording. The VQCapture provides: 1. network level statistics 2. Video Quality measurement VMOS 3. Audio Quality measurement
- 5. Receiver Side testing There are two main measurements for receiver side testing: Full payload testing after resynchronization E2E latency and jitter ReceiverTransmitter Video Encoder Audio Encoder Stream Analyzer Mux / Packetizer Video Quality testing Network Emulator Re-Synced Video comparison E2E latency & jitter