17 Apr 2017

Encode videos for Dynamic Adaptive Streaming over HTTP

Since the video tag appeared in the HTML5 specifications, it has been much more easier (and standard) to play videos in web browsers. The video tag allowing multiple sources, it's possible to have one video encoded in different format (MPEG-4, WebM…) so it can be read by all web browsers. But there's one thing it doesn't handle by default, it's having the same video in different sizes/bitrates. This is necessary to send a video according to the available bandwidth, for instance if you are on your desktop computer connected with optical fiber you could get the video with the best quality (highest bitrate), and if you are watching the video on your mobile phone with a poor 2G connection, you could get the video with the lowest bitrate.

For this to work you need a video encoded at different bitrates, split in small chunks and a video player that is able to measure the network bandwitdh and select which chunks with which bitrate to download.

Since 2009, the most common way to do that is to use Apple's HTTP Live Streaming (HLS) system, but it's not standard. At the end of 2011 the Dynamic Adaptive Streaming over HTTP (DASH) standard has been released. HLS and DASH are very similar but DASH has less constraints than HLS.

I converted the few videos I host on this blog to stream them with DASH.

Encode the video

We need te generate multiple video files at different bitrates and one audio file (multiple audio files can be used too).

Get the video metadata

With a file called video.mp4:

$ ffmpeg -i video.mp4
ffmpeg version 3.3 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.1.0 (clang-802.0.41)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libavresample   3.  5.  0 /  3.  5.  0
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp41mp42isom
    creation_time   : 2015-08-29T21:26:07.000000Z
  Duration: 00:01:29.71, start: 0.000000, bitrate: 20222 kb/s
    Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 245 kb/s (default)
    Metadata:
      creation_time   : 2015-08-29T21:26:07.000000Z
      handler_name    : Core Media Audio
    Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 19972 kb/s, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 24k tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2015-08-29T21:26:07.000000Z
      handler_name    : Core Media Video
At least one output file must be specified

That's a 1920×1080 video at 24 frames per seconds with a bitrate of 20 Mb/s and a stereo audio track at 245 kb/s.

Extract the audio track

Either extract the audio track as is:

$ ffmpeg -i video.mp4 -c:a copy -vn video-audio.mp4

Or extract the audio track and re-encode it (here with 2 channels (stereo) at 128 kb/s):

$ ffmpeg -i video.mp4 -c:a aac -ac 2 -ab 128k -vn video-audio.mp4

Extract and re-encode the video track

$ ffmpeg -i video.mp4 -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 5300k -maxrate 5300k -bufsize 2650k -vf 'scale=-1:1080' video-1080.mp4
$ ffmpeg -i video.mp4 -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 2400k -maxrate 2400k -bufsize 1200k -vf 'scale=-1:720' video-720.mp4
$ ffmpeg -i video.mp4 -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 1060k -maxrate 1060k -bufsize 530k -vf 'scale=-1:478' video-480.mp4
$ ffmpeg -i video.mp4 -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 600k -maxrate 600k -bufsize 300k -vf 'scale=-1:360' video-360.mp4
$ ffmpeg -i video.mp4 -an -c:v libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 260k -maxrate 260k -bufsize 130k -vf 'scale=-1:242' video-240.mp4

The video is encoded using H.264 codec, force to have a key frame every 24 frames (so in this case, every second, this allow to have the video segmented by chunks of 1 second in length). The bitrate is evaluated according to the buffer size, so in order to be sure the encoding is close to the requested rate, the buffer size should be lower than the rate (since the segments have a duration of 1 second).

Generate the MPD file

Now there are 1 audio file and 5 video files. A Media Presentation Description (MPD) file has to be created, this file is a sort of index referencing the different video and audio tracks with their bitrate, size and how the segments are ordered. This is the file that is loaded by the player:

$ MP4Box -dash 1000 -rap -frag-rap -profile onDemand -out video.mpd video-1080.mp4 video-720.mp4 video-480.mp4 video-360.mp4 video-240.mp4 video-audio.mp4

The dash option set the duration of each segment, 1000 ms to match the 1 second segments of the videos.

Now there are 6 new audio and video files (with dashinit in their name) and 1 MPD file. Those are the necessary files to stream the video.

You can verify the generated file with this MPD validator.

Configure the web server

In order to get the video played properly, it's better to ensure that the web server returns the MIME type application/dash+xml for MPD files. For example, in the website's Apache configuration file:

AddType application/dash+xml .mpd

Configure a player

Dash.js is the default player to play DASH videos. On my blog I use Video.js which does not handle DASH by default but there's a plugin for that.

Some drawbacks:

Dash.js only works with web browsers with Media Source Extensions (MSE), which currently means : no iOS (Apple, what are you doing?).
Video.js has to be initialized in a different way than just a plain video tag to play DASH videos.
The plugin does not allow to manually choose the bitrate.

Here is the HTML sample:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<html>
  <head>
    <script src="/js/jquery.js"></script>
    <script src="/js/video.js"></script>
    <script src="/js/dash.all.min.js"></script>
    <script src="/js/videojs-dash.js"></script>
  </head>
  <body>
    <video class="video-js" controls preload="metadata" data-dash="video.mpd" data-setup="{}">
      <source src="video.mp4" type="video/mp4" />
    </video>
    <script>
      if ('MediaSource' in window) {
        $("video").each(function() {
          $(this).children("source").remove();
          var player = videojs(this);
          var source = $(this).data("dash");
          player.ready(function() {
            player.src({
              src: source,
              type: 'application/dash+xml'
            });
          });
        });
      }
    </script>
  </body>
</html>

Lines 3 to 6: load the JavaScript libraries.
Line 9: declare the video tag, I add the location of the MPD file as a data attribute.
Line 10: set the original video as source (fallback for browsers without MSE).
Line 13: if the browser support MSE.
Line 14: for each video tag.
Line 15: remove fallback source tag (otherwise Chrome still reads it instead of the DASH video).
Line 16: initialize Video.js.
Line 17: retrieve the location of the MPD file.
Lines 19 to 22: set the MPD file as a source for this video.

It's still a bit messy, I hope it will get better soon with the new Video.js 6.

Here is the result with a highly uninteresting video (the video should play and you should see a lot of messages from Dash.js in the browser's console):