Berlin 10/10/2016
Efficient Trick Modes in MPEG-DASH
Adaptive Streaming with GStreamer
Visla Systems Ltd
Wojciech Przybyl
- New to gst community
- 10+ years experience in Linux & Media products development
- SetTopBox & DVR development
- SoC: ST / Broadcom / TI / Freescale
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
MPEG-DASH - Dynamic Adaptive Streaming over HTTP
- Codec agnostic: H.264, H.265, VP8, VP9, PCM, AAC, AC-3, DTS etc.
- Containers: MPEG-2 TS, ISOBMFF (MP4)
- Independent downloadable fragments 1-5 sec long
- Adaptive bitrate & framerate
- Streams download over HTTP 1.1
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Simple MPEG-DASH pipeline in GStreamer
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
gst-launch-1.0 souphttpsrc ! dashdemux ! qtdemux ! queue ! h264parse ! avdec_h264 ! autovideosink
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Trick Modes: Speed and Direction != x1
- Gstreamer API: gst_event_new_seek()
- Judging Trick Modes
- efficiency
- user experience
- complexity
- cost
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Simple Trick Modes
- Play it all but faster
- Download all
- Decode all
- Discard frames in a sink element
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Simple Trick Modes
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Local playback with low speed is OK
- What about the bandwidth?
- What about higher speeds: x30?
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Efficiency: None / Impossible
- User Experience: Perfect until it hits the limit
- Complexity: Simple
- Cost: Low
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Pause & Seek Trick Modes
Pause Playback →
Seek&Flush
Download
Decode
Present
← Repeat
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Pause and Seek
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Efficiency: Moderate
- User Experience: Moderate
- Complexity: Moderate
- Cost: Low
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Key frames only Trick Modes
- Download key frames only
- Perform the rest as in Simple Tick Mode
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Key frames only
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Every DASH fragment starts with a key frame
- Add sidx&ssix box and use HTTP GET RANGE request
- Should we add more I frames when reencoding?
- qtdemux is behind dashdemux
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Efficiency: High
- User Experience: Moderate / Poor
- Complexity: High
- Cost: Low
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Efficient Trick Modes
- Download separate (Sub)Representation per speed
holding only&all the frames which are going to be presented
- Perform the rest as in Simple Trick Mode
- Frames discarding is done during stream encoding
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Efficient Trick Modes
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Bandwidth & Processing Power for speed x5
is the same as at speed x1
- Image quality stays (almost) the same
- Bitrate per frame at speed x5 is the same as at speed x1
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Image quality stays (almost) the same
- Why almost?
- True for speed x1 vs x5
- Almost true for speed x1 vs x30 → Stream is I frames only
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Status
- Proof of Concept stage done
- Choosing correct (Sub)Representation might be tricky
- Manifest's attributes:
@bandwidth
@frameRate
@MaxPlayoutRate
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
- Efficiency: Perfect
- User Experience: Perfect
- Complexity: Moderate
- Cost: High
Cost = (Storage+Encoding+Packaging+Encryption) * N bitrates * M fps substreams
+Delivery
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
DEMO
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Questions and (hopefully) Answers
ABOUT
VISLA SYSTEMS
ADAPTIVE
STREAMING
CLASSIC
TRICK MODES
EFFICIENT
TRICK MODES
DEMO Q & A
Thank you
Wojciech Przybyl
wojciech@visla.co.uk

Wojciech Przybyl - Efficient Trick Modes with MPEG-DASH.pdf

  • 1.
    Berlin 10/10/2016 Efficient TrickModes in MPEG-DASH Adaptive Streaming with GStreamer Visla Systems Ltd Wojciech Przybyl
  • 2.
    - New togst community - 10+ years experience in Linux & Media products development - SetTopBox & DVR development - SoC: ST / Broadcom / TI / Freescale ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 3.
    MPEG-DASH - DynamicAdaptive Streaming over HTTP - Codec agnostic: H.264, H.265, VP8, VP9, PCM, AAC, AC-3, DTS etc. - Containers: MPEG-2 TS, ISOBMFF (MP4) - Independent downloadable fragments 1-5 sec long - Adaptive bitrate & framerate - Streams download over HTTP 1.1 ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 4.
    Simple MPEG-DASH pipelinein GStreamer ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 5.
    gst-launch-1.0 souphttpsrc !dashdemux ! qtdemux ! queue ! h264parse ! avdec_h264 ! autovideosink ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 6.
    - Trick Modes:Speed and Direction != x1 - Gstreamer API: gst_event_new_seek() - Judging Trick Modes - efficiency - user experience - complexity - cost ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 7.
    Simple Trick Modes -Play it all but faster - Download all - Decode all - Discard frames in a sink element ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 8.
    Simple Trick Modes ABOUT VISLASYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 9.
    - Local playbackwith low speed is OK - What about the bandwidth? - What about higher speeds: x30? ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 10.
    - Efficiency: None/ Impossible - User Experience: Perfect until it hits the limit - Complexity: Simple - Cost: Low ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 11.
    Pause & SeekTrick Modes Pause Playback → Seek&Flush Download Decode Present ← Repeat ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 12.
    Pause and Seek ABOUT VISLASYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 13.
    - Efficiency: Moderate -User Experience: Moderate - Complexity: Moderate - Cost: Low ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 14.
    Key frames onlyTrick Modes - Download key frames only - Perform the rest as in Simple Tick Mode ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 15.
    Key frames only ABOUT VISLASYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 16.
    - Every DASHfragment starts with a key frame - Add sidx&ssix box and use HTTP GET RANGE request - Should we add more I frames when reencoding? - qtdemux is behind dashdemux ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 17.
    - Efficiency: High -User Experience: Moderate / Poor - Complexity: High - Cost: Low ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 18.
    Efficient Trick Modes -Download separate (Sub)Representation per speed holding only&all the frames which are going to be presented - Perform the rest as in Simple Trick Mode - Frames discarding is done during stream encoding ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 19.
    Efficient Trick Modes ABOUT VISLASYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 20.
    - Bandwidth &Processing Power for speed x5 is the same as at speed x1 - Image quality stays (almost) the same - Bitrate per frame at speed x5 is the same as at speed x1 ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 21.
    - Image qualitystays (almost) the same - Why almost? - True for speed x1 vs x5 - Almost true for speed x1 vs x30 → Stream is I frames only ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 22.
    Status - Proof ofConcept stage done - Choosing correct (Sub)Representation might be tricky - Manifest's attributes: @bandwidth @frameRate @MaxPlayoutRate ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 23.
    - Efficiency: Perfect -User Experience: Perfect - Complexity: Moderate - Cost: High Cost = (Storage+Encoding+Packaging+Encryption) * N bitrates * M fps substreams +Delivery ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 24.
  • 25.
    Questions and (hopefully)Answers ABOUT VISLA SYSTEMS ADAPTIVE STREAMING CLASSIC TRICK MODES EFFICIENT TRICK MODES DEMO Q & A
  • 26.