How to Configure (Unified Origin)

Note

Our software requires a license key. How to specify your license key for Apache is explained as part of Setting up a virtual host (and specifying your license key) on this page. For more info, see License Key.

Basic Apache configuration on Linux

After you have installed our software and Apache on your Linux distribution of choice, you still need to configure several things before you can successfully stream video. First of all, you will need to make sure the correct Apache modules are enabled:

For more info on loading modules into Apache, please see LoadModule.

After you have correctly configured the modules, setting up a virtual host (which includes specifying the license key that Apache needs to load) will be the final step before you have a fully working video streaming setup: Setting up a virtual host (and specifying your license key).

Ensure 'headers' module is enabled

Amazon Linux 2 / CentOS / RedHat

First, check if headers module is enabled or not (if headers_module is not present in the list that is printed, the module is not enabled):

sudo apachectl -M | sort

If headers module is not enabled enable it by opening /etc/httpd/conf.modules.d/00-base.conf in a text editor to uncomment the line for headers_module.

Change this:

#LoadModule headers_module modules/mod_headers.so

To this:

LoadModule headers_module modules/mod_headers.so

And save (please note these changes will only take effect after a restart of Apache).

Debian / Ubuntu

First, check if headers module is enabled or not:

sudo a2query -m headers

If headers module is not enabled, enable it.

sudo a2enmod headers

These changes will only take effect after a restart of Apache.

Set MPM model to 'worker', not 'event'

On most Linux distributions, the default MPM model in Apache 2.4 is 'event'. Please make sure this is changed to either 'worker' or 'prefork'. See the Apache documentation for more information on MPM models.

Amazon Linux 2 / CentOS / RedHat

First, check which MPM model Apache is currently using:

sudo apachectl -M | sort

If Apache is using the MPM model 'event', change it to 'worker' (or 'prefork') by opening /etc/httpd/conf.modules.d/00-mpm.conf in a text editor to uncomment the line for mpm_worker_module and comment out the one for mpm_event_module:

#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

#LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule mpm_worker_module modules/mod_mpm_worker.so

Then save (please note these changes will only take effect after a restart of Apache).

Debian / Ubuntu

First, check which MPM model Apache is currently using:

sudo a2query -M

Then, Apache is using the MPM model 'event', change it to 'worker' (or 'prefork'):

sudo a2dismod mpm_event
sudo a2enmod mpm_worker

Note, these changes will only take effect after a restart of Apache.

Enable 'mod_smooth_streaming'

If you have just installed the 'mod_smooth_streaming' module, you still need to enable it.

Amazon Linux 2 / CentOS / RedHat

If you have just installed the 'mod_smooth_streaming' module, you still need to enable it. Checking whether it is already installed is unnecessary, but, in case you ever want to, you can do so like this (if smooth_streaming_module is not present in the list that is printed, the module is not enabled):

sudo apachectl -M | sort

To enable the module, open /etc/httpd/conf/httpd.conf in a text editor and add the following line:

LoadModule smooth_streaming_module modules/mod_smooth_streaming.so

If you are going to use subrequests (see :ref:subrequest - todo) and require Amazon S3 authentication, you must also enable the mod_unified_s3_auth module, by adding:

LoadModule unified_s3_auth_module modules/mod_unified_s3_auth.so

If you are going to use manifest-edit from the Origin as an ext_filter (see Enabling Manifest Edit), you must also enable the mod_ext_filter module, by adding:

LoadModule ext_filter_module modules/mod_ext_filter.so

And save (please note these changes will only take effect after a restart of Apache).

Debian / Ubuntu

If you have just installed the 'mod_smooth_streaming' module, you still need to enable it. Although unnecessary if you wish to check the module is installed you can do so like this (if smooth_streaming_module is not present in the printed list, the module is not enabled):

sudo apache2ctl -M | sort

Note

Do not use a2query to check whether 'mod_smooth_streaming' is enabled, because the module will not be listed in the results even if it is active.

To enable the module:

sudo a2enmod mod_smooth_streaming

If you are going to use subrequests (see :ref:subrequest - todo) and require Amazon S3 authentication, you must also enable the mod_unified_s3_auth module, by running:

sudo a2enmod mod_unified_s3_auth

If you are going to use manifest-edit from the Origin as an ext_filter (see Enabling Manifest Edit), you must also enable the mod_ext_filter module, by adding:

sudo a2enmod ext_filter

Note, these changes will only take effect after a restart of Apache.

Sample Apache Configuration File

The follow configuration is a example of how unified origin can be configured but would likely need to be updated to support relevant use-case or requirements.

Note

Due to the use of IfDefine, it is necessary to start apache with the -D parameter flag to ensure the variables are populated is a value

# builtin modules
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
# Unified Streaming modules
LoadModule smooth_streaming_module modules/mod_smooth_streaming.so
LoadModule unified_s3_auth_module modules/mod_unified_s3_auth.so

AddHandler smooth-streaming.extensions .ism .isml

ServerName unified-origin

UspLicenseKey /etc/usp-license.key

LogFormat '${LOG_FORMAT}' log_format

# Origin virtualhost
<VirtualHost 0.0.0.0:81>

</VirtualHost>

<VirtualHost 0.0.0.0:80>
# don't log kubernetes probes
  SetEnvIf User-Agent "kube-probe/1\.18" dontlog
  CustomLog /dev/stdout log_format env=!dontlog
  ErrorLog /dev/stderr

  LogLevel ${LOG_LEVEL}
  SSLProxyEngine on

  DocumentRoot /var/www/unified-origin

  Header set Access-Control-Allow-Headers "origin, range"
  Header set Access-Control-Allow-Methods "GET, HEAD, OPTIONS"
  Header set Access-Control-Allow-Origin "*"
  Header set Access-Control-Expose-Headers "Server,range"

  # Enable Origin and use subrequests instead of libcurl
  <Location />
    UspHandleIsm on
    UspEnableSubreq on
  </Location>

  # Set Cache-Control value for VOD/Catchup client manifest if not set.
  # By default Unified-Origin only sets Cache-Control Headers for Live
  # therefore it maybe necessary to set this when requesting VOD/catchup clips
  # from a isml.
  <LocationMatch "(\.mpd|\.m3u8|Manifest)$">
    Header setifempty Cache-Control "max-age=2"
  </LocationMatch>

  # Manifest Edit configuration
  Define ManifestEditCmdMpd "/usr/bin/manifest_edit --python_pipeline_path=/usr/share/manifest-edit/mpd -v 2 -o stdout: stdin:"
  Define ManifestEditCmdM3u8 "/usr/bin/manifest_edit --python_pipeline_path=/usr/share/manifest-edit/m3u8 -v 2 -o stdout: stdin:"
  ExtFilterDefine pipelineMpd mode=output cmd="${ManifestEditCmdMpd}"
  ExtFilterDefine pipelineM3u8 mode=output cmd="${ManifestEditCmdM3u8}"

  <LocationMatch ".*\.ism[l]?$">
    <If "%{QUERY_STRING} =~ /python_pipeline_config=/">
      ExtFilterOptions LogStdErr Onfail=abort
      <If "%{QUERY_STRING} =~ /file=\.mpd/">
        SetOutputFilter pipelineMpd
      </If>
      <ElseIf "%{QUERY_STRING} =~ /file=\.m3u8/">
        SetOutputFilter pipelineM3u8
      </ElseIf>
    </If>
  </LocationMatch>

  # remote storage configuration
  <IfDefine REMOTE_STORAGE_URL>
    <Location "/${REMOTE_PATH}/">
      IsmProxyPass "${REMOTE_STORAGE_URL}"
    </Location>

    <Proxy "${REMOTE_STORAGE_URL}">
      ProxySet connectiontimeout=5 enablereuse=on keepalive=on retry=0 timeout=30 ttl=300
      RequestHeader unset Accept-Encoding
      <IfDefine S3_ACCESS_KEY>
        S3AccessKey ${S3_ACCESS_KEY}
      </IfDefine>
      <IfDefine S3_SECRET_KEY>
        S3SecretKey ${S3_SECRET_KEY}
      </IfDefine>
      <IfDefine S3_REGION>
        S3Region ${S3_REGION}
      </IfDefine>
    </Proxy>
  </IfDefine>

</VirtualHost>

<Directory /var/www/unified-origin>
  Require all granted
</Directory>

Basic Apache configuration on Windows

After you have installed Unified Origin on Windows, the three directories we will be focussing on for the configuration of Apache are:

Directory Importance
C:\Apache24\modules Location of mod_smooth_streaming.so
C:\Apache24\conf Location of httpd.conf (general config)
C:\Apache24\conf\extra Location of httpd-vhosts.conf (virtual host)

First, we will enable the 'headers' module by using a plain-text editor to uncomment the below line in httpd.conf:

#LoadModule headers_module modules/mod_headers.so

So that it looks like this:

LoadModule headers_module modules/mod_headers.so

Then we will add the following line to httpd.conf to instruct Apache to load the mod_smooth_streaming.so:

LoadModule smooth_streaming_module modules/mod_smooth_streaming.so

If you are going to use subrequests (see :ref:subrequest - todo) and require Amazon S3 authentication, you must also enable the mod_unified_s3_auth module, by adding:

LoadModule unified_s3_auth_module modules/mod_unified_s3_auth.so

If you are going to use manifest-edit from the Origin as an ext_filter (see Enabling Manifest Edit), you must also enable the mod_ext_filter module, by adding:

LoadModule ext_filter_module modules/mod_ext_filter.so

Finally, in httpd.conf uncomment the second line shown below:

#Virtual hosts
#Include conf/extra/httpd-vhosts.conf

So that it will look like this (and will tell Apache to include the virtual host configuration specified in httpd-vhosts.conf):

#Virtual hosts
Include conf/extra/httpd-vhosts.conf

Attention

The DocumentRoot for the default virtual host (httpd-vhosts.conf) needs to be updated before you can access any files.

Setting up a virtual host (and specifying your license key)

After the Basic Apache configuration on Linux or Basic Apache configuration on Windows, you will need to set up a virtual host in Apache to complete a working video streaming setup.

The configurations below apply to both the Windows and the Linux platform, though the exact location of the Virtual Host configuration will differ for both of these platforms and between different Linux distributions. For more information on configuring virtual hosts, please consult Apache's official documentation on the VirtualHost directive.

If you need more fine grained control, please see the 'More detailed configuration' below. To ensure you have a bare minimum of security in place, please read through the Apache documentation's security tips.

Note

If you want to make use of multiple virtual hosts on CentOS, you need to add the following to /etc/httpd/httpd.conf:

NameVirtualHost *:80

Specifying your license key

We will first specify our license key in our virtual host file. Note this must be done outside of the actual virtual host configuration. How you specify this differs slightly between Linux and Windows, both are explained below. This is the only part where considerable differences exist between setting up your virtual host, the rest of the configuration can be considered universal.

Specifying license on Linux

To specify your license key on Linux, add the UspLicenseKey directive to the file containing your virtual host configuration, but add it outside of the configuration of the virtual host itself. Use the directive to point to the file in which you have stored your license key as shown below (where </path/to/usp-license.key> should be replaced to point to the correct location):

UspLicenseKey </path/to/usp-license.key>

<VirtualHost *:80>

  # Rest of config

</VirtualHost>

Specifying license on Windows

To specify your license key on Windows, first create a file called usp-license.conf inside the C:\Apache24\conf directory. This file should contain the UspLicenseKey directive and your full license key. It should look like so (where <your-actual-license-key> should be replaced by your actual license key):

UspLicenseKey <your-actual-usp-license-key>

Then you need use the Include directive inside the file containing your virtual host configuration (httpd-vhosts.conf). Make sure to add this directive outside of the configuration of the virtual host itself, and use it to include the contents of the file you have just created into the configuration:

Include "conf/usp-license.conf"

<VirtualHost *:80>

  # Rest of config

</VirtualHost>

'UspHandleIsm'

Now that you have specified your license key, we can start configuring the actual virtual host. This guide will focus specifically on those parts related to setting up Unified Origin. More general settings such as ServerName, DocumentRoot and the configuration of your logs are not covered here.

To enable Unified Origin on a given virtual hosti use UspHandleIsm.

The UspHandleIsm directive is used in combination with the <Location> directive to specify for which locations the smooth_streaming_module should be activated. If you want to always have the module handle the requests that it supports, you can simply add a single location to the virtual host's configuration like so:

<Location />
  UspHandleIsm on
</Location>

Important

The UspHandleIsm configuration token MUST be specified in a <Location> directive, not in a <Directory> directive.

Cross domain access (CORS headers)

Another thing that you will need to configure as part of your virtual host configuration, is setting the Cross-Origin Resource Sharing (CORS) headers. These headers are necessary for HTML5 based playout scenarios in which the (Javascript) player and the content are hosted on different domains. If these headers are not present in such a scenario, playout won't work. To set the CORS header correctly, simply add the following lines to Unified Origin's virtual host configuration:

# Necessary for Media Source Extensions (MSE)
Header always set Access-Control-Allow-Headers "origin, range"
Header always set Access-Control-Allow-Methods "GET, HEAD, OPTIONS"
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Expose-Headers "Server,range"

When the player and the content are hosted on different domains, similar information needs to be communicated for playout scenarios using Flash (Adobe HTTP Dynamic Streaming) or Silverlight (Microsoft HTTP Smooth Streaming). For such playout scenarios to work, two files need be hosted in virtual host's DocumentRoot. Preconfigured versions of those files can be downloaded here: crossdomain.xml (for HDS) and clientaccesspolicy.xml (for Smooth).

Additional settings for Live streaming

When using the webserver module for ingesting live streams, you need to take various timeout settings into account.

Since a long-running HTTP POST is used to transfer the audio/video from the encoder to the webserver, we don't want Apache to close the connection when certain timeouts are reached.

Make sure and verify that LimitRequestBody is set to 0 and in the case of using mod_reqtimeout that the timeout values are sane. In case of the latter, please note that you need to update the global RequestReadTimeout values.

Overriding the values in a virtual host does not work.

LimitRequestBody 0
<IfModule reqtimeout_module>
   RequestReadTimeout header=0 body=0
</IfModule>

More detailed configuration

If you need to control the functionality in more detail you can use the regular expression LocationMatch to enable or disable functionality.

Enable just-in-time packaging for VOD (.ism) and Live (.isml):

<LocationMatch "\.[is]sml?/">
  UspHandleIsm on
</LocationMatch>

Enable just-in-time packaging for MP4:

<LocationMatch "\.mp4/">
  UspHandleIsm on
</LocationMatch>

The module can also handle requests for files packaged with the F4FPackager (mod_f4fhttp). To enable this you have to add the following configuration to rewrite files ending in SegNNN-FragNNN with mod_smooth_streaming:

<LocationMatch "Seg[\d]+-Frag[\d]+$">
  UspHandleF4f on
</LocationMatch>

Serve .mp4 files through Unified Origin. This may be useful for playout of statically stored dref-ed MP4 files and/or playout of pseudo streaming (mod_smooth_streaming).

<LocationMatch "\.mp4$">
  SetHandler smooth-streaming.extensions
</LocationMatch>

Please note that Apache's Alias directive does not mix with the Origin module. Instead of using the Alias directive, you may want to use a Virtual Host with the DocumentRoot set to the directory-path of the Alias.

Prevent the download of media files

In order to prevent the download of (source) mediafiles, add the following to your webserver config.

<LocationMatch "\.(ismv|isma|ismt)$">
  Require all denied
</LocationMatch>

When the source content is MP4 then the rule can be adapted to mp4 as well. The sampe applies to CMAF as source (cmfv, cmfa and cmft).

Prevent the download of server manifests

In order to protect your server manifests (VOD and Live) you have to disable access to ism and isml files.

# Block access to server manifests when the module is loaded
AddHandler smooth-streaming.extensions .ism .isml

# For when the module is not loaded
<IfModule !smooth_streaming_module>
  <LocationMatch "\.isml?$">
    Require all denied
  </LocationMatch>
</IfModule>

Enabling Manifest Edit

Manifest Edit functionalities in the Origin are available through the use of mod_ext_filter.

Specifically for this use case, the manifest_edit command line tool supports a query argument python_pipeline_config, to indicate the yaml to use. The query argument is passed in via Apache's QUERY_STRING environment variable.

Basic Settings

The following example illustrates how you would configure the activation of a my_use_case.yaml use case. This file would typically be located in a folder of your choice, where you keep your pipeline configuration files. We are going to assume that this is the /home/user/my_use_cases folder on Linux and the C:/my_use_cases folder on Windows. You should of course modify these values to reflect your specific configuration. We also assume that the UspLicenseKey variable would point to your license file on disk (see Specifying your license key).

Based on the above assumptions, this is the section that should be present in your Apache configuration to define a filter based on your pipeline:

#### Customize these values with your specific folder/file names #####
<IfDefine !WIN32>
  Define ManifestEditConfDir /home/user/my_use_cases
</IfDefine>
<IfDefine WIN32>
  Define ManifestEditConfDir "C:/my_use_cases"
</IfDefine>

Define MyPipelineConf    ${ManifestEditConfDir}/my_use_case.yaml
######################################################################

<IfDefine !WIN32>
  Define ManifestEditExe /usr/bin/manifest_edit
</IfDefine>
<IfDefine WIN32>
  Define ManifestEditExe "C:/Program\ Files/Unified\ Streaming/manifest_edit.bat"
</IfDefine>

Define ManifestEditCmd  "'${ManifestEditExe}' --license_key=${UspLicenseKey} --python_pipeline_path=${ManifestEditConfDir} -v 2 -o stdout: stdin:'"

ExtFilterDefine my_pipeline mode=output cmd="${ManifestEditCmd}"

With the above configuration in mind, you can now activate the ext_filter module with any standard Apache matching rules. The recommended way is to use a LocationMatch clause to enable Manifest Edit only for the "manifest" URIs (i.e. .ism/.mpd for dash or .ism/.m3u8 for HLS main playlists).

Apache will pass query parameters to the manifest edit executable, who will look for the presence of a python_pipeline_config argument and use it as the name of the yaml configuration file to load. So, the ext filter pipeline should only be invoked when such a query parameter is present. This can be done with the following Apache configuration section:

<LocationMatch ".*\.[is]sm[l]?/\.mpd$">
  <If "%{QUERY_STRING} =~ /python_pipeline_config=/">
      SetOutputFilter my_pipeline
      ExtFilterOptions LogStdErr Onfail=abort
  </If>
</LocationMatch>

<LocationMatch ".*\.[is]sm[l]?/\.m3u8$">
  <If "%{QUERY_STRING} =~ /python_pipeline_config=/">
      SetOutputFilter my_pipeline
      ExtFilterOptions LogStdErr Onfail=abort
  </If>
</LocationMatch>

Warning

Whatever method you choose to associate a given URI to a manifest edit pipeline, make sure that the rule matches only URIs for a manifest and not for any media segment. This can be accomplished in general by checking for the presence in the URI of the manifest extension (i.e. .mpd for Dash or .m3u8 for HLS). Failure to do so may result in a broken stream.

This should be all you need to use manifest edit, that can now be invoked by simply passing the name of the desired pipeline configuration file via the python_pipeline_config query parameter. Following our example, to have a manifest be edited by the pipeline described by the my_use_case.yaml configuration file, you will simply use one of the following URLS:

http://my-origin.com/my-video.ism/.mpd?python_pipeline_config=my_use_case (for DASH)
http://my-origin.com/my-video.ism/.m3u8?python_pipeline_config=my_use_case (for HLS)

The main advantage of this configuration is that Apache needs only to be configured once, to provide the folder where your pipeline configuration files are. When new configuration files are added or removed or renamed, the configuration doesn't need to be changed and new pipelines are just selected via query parameters.

Warning

Make sure the query parameter refers to the base configuration file name without file extension. I.e. to activate the pipeline described in my_use_case.yaml, use the ?python_pipeline_config=my_use_case query parameter.

Using remote storage with Manifest Edit

When either the server manifest or the media segments are on a remote object storage, an additional configuration is required. In particular, a separate virtual host is needed, dedicated to handle remote storage. A reverse proxy configuration will map any request for a remote path of choice to the storage server.

In order to do so, you will need to add an additional Virtual Host section to your origin configuration and a new listen port (i.e. 81) to your main configuration. The new Virtual Host will act as a "storage backend proxy".

Assuming that {{REMOTE_STORAGE_URL}} is your remote storage's URL, the following configuration will enable the storage backend proxy:

<VirtualHost *:81>
ServerName unified-origin-backend

ErrorLog /var/log/apache2/origin-storage-error.log
CustomLog /var/log/apache2/origin-storage-access.log combined
LogLevel warn

<Location "/">
  UspHandleIsm on
  UspEnableSubreq on
  IsmProxyPass {{REMOTE_STORAGE_URL}}
</Location>

<Proxy "{{REMOTE_STORAGE_URL}}">
  SSLProxyEngine on
  ProxySet connectiontimeout=5 enablereuse=on keepalive=on retry=0 timeout=30 ttl=300
</Proxy>

</VirtualHost>

Once the new virtual host is in place, you should make sure the origin uses it (only) when accessing a path pointing to remote storage. Assuming this is {{REMOTE_PATH}}, you should then add the following to your origin virtual host config:

# Location for streaming local server manifests
<Location "/">
  UspHandleIsm on
  UspEnableSubreq on
</Location>

# Location for streaming remote manifests via port 81 vhost
<Location "/{{REMOTE_PATH}}/">
  ProxyPass http://0.0.0.0:81/
  ProxyPassReverse http://0.0.0.0:81/
</Location>

If you need more details about this particular configuration, a complete working example is available in the Manifest Edit docker image github project.

Note

If you are running Apache on Windows, we recommend using the Apache Lounge builds from https://www.apachelounge.com/download/.

From that site, please make sure that you use at least Apache 2.4.46, with a build date of "02 Oct '20" or later, as this includes a crucial fix for the use of external filters on Windows.

See also this Apache Lounge forum topic, and this Apache bug report.

Encoder specific settings

Specifics for Expression Encoder 4

If you plan on encoding live streams with Expression Encoder 4 you also have to configure what you return as the Server HTTP response Header. The default is Full, which means that information about the OS-type and the compiled in modules will be send. On Debian / Ubuntu you can find and configure this directive in /etc/apache2/conf.d/security:

ServerTokens Full

Specifics for Media Excel HERO

The HERO encoder requires KeepAlive in the HTTP responses. If it's not present then it closes the connection and doesn't start sending the stream. Look for the KeepAlive directive in the Apache configuration and change it to 'On'.

KeepAlive On