Dynamic Manifests

Overview

Dynamic manifests specify a proxy for Unified VOD/LIVE for fetching the server manifest file. This is useful for dynamically updating options like DRM settings, encryption keys, enable/disable playout that are managed by your CMS.

Schematically:

          GET
  [CMS]   <--   [origin]  <-- client request
video.ism     IsmProxyPass

Apache

New in version 1.10.28.

For Apache the IsmProxyPass directive can be specified in a Directory or Location:

<Directory "/var/www/video/proxy" >
  IsmProxyPass http://other-server/
</Directory>

or:

<Location "/proxy" >
  IsmProxyPass http://other-server/
</Location>

Note

In the following section we will use the new <Location> everywhere as that is a better match with Apache - as outlined in Adding UspEnableSubreq directives.

How it works

When the IsmProxyPass configuration is present, the webserver module requests the server manifest file from the server specified by the URL.

For example, a request for:

http://www.example.com/video/proxy/tears-of-steel/tears-of-steel.ism/.m3u8

would normally read the server manifest file from:

/var/www/video/proxy/tears-of-steel/tears-of-steel.ism

but with IsmProxyPass enabled it fetches it from:

http://other-server/tears-of-steel/tears-of-steel.ism/.m3u8

and all URLs with 'proxy' in the path will map to the other-server.

The trailing / in the URL used for IsmProxyPass is mandatory.

The directive must be in the vhost config: /etc/apache2/sites-enabled/vhost.conf and not the global webserver config: /etc/apache2/apache2.conf.

The 'proxy' directory (or any other name used) should not exist on the webserver because it is a virtual path.

For example using /etc/apache2/sites-enabled/usp-evaluation.conf:

<Location "/proxy">
  IsmProxyPass http://other-server/
</Location>

So the '/var/www/usp-evaluation' directory exists but the '/var/www/usp-evaluation/proxy' directory does not and maps to http://other-server.

This applies to all examples on this page.

Important

There is limitation on the number of TCP connections that can be open, based on the range of available ports. By default, Linux's set of outgoing ports is something like 31000 - 61000. This should be taken into consideration because when a connection is closed, ports stay reserved in TIME_WAIT status for 60 seconds by default and connections cannot be re-used by Origin. This means that the server may run out of available ports when it needs to make many outgoing requests to serve a large number of incoming requests, which is a plausible scenario when using IsmProxyPass, especially in combination with a proxy cache on the same server. This problem can be solved by distributing requests across more servers, increasing the port range (sysctl net.ipv4.ip_local_port_range), or lowering the timeout (sysctl net.ipv4.tcp_fin_timeout).

Playback content from other domains

To provide a more elaborate example, let us assume we have content (an MP4 video) hosted at: http://content.bitsontherun.com/videos/3XnJSIm4-kNspJqnJ.mp4 and we want to make this available in the HTTP Smooth Streaming format.

The first step is to set up IsmProxyPass to create a server manifest file dynamically for any given audio/video URLs.

<Location "/var/www/usp/ism">
  IsmProxyPass http://demo.unified-streaming.com/smil.php/
</Location>

The webserver module now requests the server manifest file using the given smil.php script for any URL that starts with http://demo.unified-streaming.com/ism.

If you were requesting the Smooth Streaming client manifest file for the MP4 video, the example URL is:

http://demo.unified-streaming.com/ism/CACHE_ID.ism/Manifest?url=http%3A%2F%2Fcontent.bitsontherun.com%2Fvideos%2F3XnJSIm4-kNspJqnJ.mp4

The internal URL that the webserver uses to fetch the server manifest file becomes:

http://demo.unified-streaming.com/smil.php/CACHE_ID.ism?url=http%3A%2F%2Fcontent.bitsontherun.com%2Fvideos%2F3XnJSIm4-kNspJqnJ.mp4

The following PHP script generates a smil file with the audio/video tracks pointing to the original content:

<?php
error_reporting(0);
$url = $_GET["url"];
Header('Content-Type: text/xml');
echo "<?xml version=\"1.0\" encoding=\"utf-8\"?>"
?>
<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
  </head>
  <body>
    <switch>
      <audio src="<?=htmlspecialchars($url, ENT_QUOTES, 'UTF-8')?>" systemBitrate="1"/>
      <video src="<?=htmlspecialchars($url, ENT_QUOTES, 'UTF-8')?>" systemBitrate="1"/>
    </switch>
  </body>
</smil>

Please note that the string 'CACHE_ID' in the above example is just a placeholder, it can be anything (or even removed) and may be used for internal tracking (session id etc).

Re-stream content from any domain

This is an example of a generic setup to re-stream any content from any domain.

It enables you to make any content available in all playout formats, without any requirements on the webserver running on the proxied domain (i.e. it can be a plain webserver, without support for HDS, HSS or HLS playout capabilities).

First, set up the IsmProxyPass to proxy any ingest supported content, e.g. MP4 (.mp4/.smil), HDS (.f4m) or HSS/MPEG-DASH (.ism).

<Location "/var/www/usp/direct">
  IsmProxyPass http://
</Location>

If you were requesting the Smooth Streaming client manifest file for the MP4 video the URL is:

http://demo.unified-streaming.com/direct/other.domain.com/videos/video.mp4/Manifest

The webserver running at http://other.domain.com is a plain webserver and the Origin running at http://demo.unified-streaming.com uses the URL http://other.domain.com/videos/video.mp4 to create the manifest file.

Supported formats .mp4, .ism, .smil and .f4m can be ingested from any webserver without having to run any additional media server side components on the proxied servers.

An example using content from AWS S3 is:

For DASH playout:

http://demo.unified-streaming.com/direct/usp-s3-storage.s3.amazonaws.com/tears-of-steel/tears-of-steel.ism/.mpd

or HLS:

http://demo.unified-streaming.com/direct/usp-s3-storage.s3.amazonaws.com/tears-of-steel/tears-of-steel.ism/.m3u8