Playout Control

Using different manifests

Sometimes devices cannot play certain types of video or audio, mostly because there is no support for a certain codec. However, you do want the codec for other devices, so packaging all the video in one server manifest may not the best approach.

It is possible to create more than one server manifest to cover a broad range of devices. When creating several server manifests you can select the video and audio supported for the targeted platform. Each manifest can be addressed as a separate URL, so you can present each device the content it can play.

Let's say we are targeting the following devices to deliver the same presentation, but with different technical specifications.

The targeted platforms for this example are:

Target platform

Specifications

Smartphones (iPhone, Android, Windows Phone)

Low-end devices supporting AVC Baseline profile and AAC-LC audio

Tablets (iPad, Android, Windows)

Devices supporting AVC Main profile

Gaming consoles (Xbox, PlayStation)

Higher quality video and support for Dolby Digital audio

STB, HbbTV, SmartTV

Highest quality audio and video plus support for DTS audio

Now that we have defined our target platforms, we can start packaging for these platforms. Shown below are a couple of example commands selecting certain video or audio tracks for the targets.

These are the files we are going to use in our examples:

Files

Video

Audio

video01.ismv

H.264 video 350 kbps Baseline profile

AAC-LC audio 96kbps

video02.ismv

H.264 video 700 kbps Baseline profile

video03.ismv

H.264 video 1000 kbps Baseline profile

AAC-LC audio 128kbps

video04.ismv

H.264 video 1500 kbps Main profile

video05.ismv

H.264 video 2000 kbps Main profile

video06.ismv

H.264 video 3000 kbps Main profile

AAC-LC audio 256kbps

audio01.isma

Dolby Digital audio 384 kbps

audio02.isma

DTS Audio audio 384 kbps

Scenario 1 (Smartphones)

For this targeted platform we use the first three videos with a maximum bandwidth of 1000 kbps. These devices only support the AVC Baseline profile. For audio we pick the audio of the first video, and only select the video track from the third video file.

#!/bin/bash

mp4split -o smartphone.ism \
  video01.ismv \
  video02.ismv \
  video03.ismv --track_type=video

Scenario 2 (Tablets)

For this targeted platform we use the first four videos with a maximum bandwidth of 1500 kbps. For audio we pick the audio of the 3rd video, and only select the video track from the 1st video.

#!/bin/bash

mp4split -o tablet.ism \
  video01.ismv --track_type=video \
  video02.ismv \
  video03.ismv \
  video04.ismv

Scenario 3 (Gaming consoles)

For these targeted platforms we add all the video streams except for the video with the lowest bitrate. For audio we pick the audio of the third video, and only select the video track from the sixth video file. Also since there is support for Dolby Digital, we also add these to our manifest.

#!/bin/bash

mp4split -o console.ism \
  video02.ismv \
  video03.ismv \
  video04.ismv \
  video05.ismv \
  video06.ismv --track_type=video \
  audio01.isma

Scenario 4 (STBs, HbbTV, SmartTVs)

Similar to the third scenario, but we also add DTS audio to our manifest.

#!/bin/bash

mp4split -o stb.ism \
  video02.ismv \
  video03.ismv --track_type=video \
  video04.ismv \
  video05.ismv \
  video06.ismv \
  audio01.isma \
  audio02.isma

Using dynamic track selection

New in version 1.6.1.

Requesting a manifest/playlist by default returns all the available streams. It is the player that can make the best decision on which stream to play. Sometimes it is useful to limit the number of streams, or cap the minimum/maximum bitrates returned in the manifest (e.g., for testing, specific devices, or to allow more users to watch the same video but at an average lower bitrate).

A generic way to filter the tracks is to write an expression. The expression is evaluated for each track specified in the server manifest file. Only when it evaluates to true the track is added to the returned manifest/playlist.

The expression is passed as a query parameter using the syntax filter=EXPRESSION (or using --track_filter when running mp4split), with syntax and semantics similar to standard C:

  • Expressions are evaluated for each track specified in server manifest, against track properties defined in server manifest (i.e., do not filter based on track properties as signaled clients manifests, since they might differ from properties in server manifest)

  • Relational operators like == and != take precedence over || and &&

  • Variable names to filter on and their values are case sensitive

  • To avoid unexpected results, do not rely on precedence between && and || (instead, use parentheses to make precedence explicit)

  • Use parentheses ( and ) to make expressions clear and easy to understand

Note

It's not possible to filter out Timed Metadata using a dynamic track filter. Passthrough of Timed Metadata can be enabled or disabled using the --timed_metadata option.

Some examples:

Filer Expression

Description

true

Expression evaluates to true for all tracks, so client manifest will contain all tracks.

type != "video" || systemBitrate < 400000

Include all non-video (i.e., audio and text) tracks, and only include video tracks with a bitrate lower than 400 kbps.

type == "video" || systemLanguage == "eng"

Include all video tracks, as well as non-video (i.e., audio and text) tracks that specify English language.

type != "video" || AVC_PROFILE == AVC_PROFILE_BASELINE

Include all non-video (i.e., audio and text tracks), and only video tracks that use the AVC Baseline profile.

(FourCC == "AACL" && SampleRate == 48000) || (FourCC == "AVC1" && AVC_LEVEL >= 31)

Include all AAC-LC audio tracks with a sample rate of 48KHz and all AVC video tracks with a minimum level of 3.1.

The following constants are available:

Constant variable name

Value

AVC_PROFILE_BASELINE

66

AVC_PROFILE_MAIN

77

AVC_PROFILE_HIGH

100

The following track variables are available:

Track variable name

Value

Note

type

Track's type: "audio", "video", "textstream", "data"

FourCC

Track's FourCC code, e.g., "AVC1", "AACL", "TTML", "JPEG", "dtse", "ac-3", "hvc1"

Codecs supported by Microsoft Smooth Streaming use FourCC defined by that specification, others follow ISO BMFF specification

trackID

Track's ID as specified in server manifest

Use not recommended

AudioTag

Track's format tag of the audio

Channels

Track's number of audio channels

Can only be applied to audio tracks

MaxWidth

Track's coded width (in pixels), which does not necessarily equal display width

Can only be applied to video tracks and use not recommended, rely on DisplayWidth instead

trackName

Track's name as specified in server manifest

Use not recommended

MaxHeight

Track's coded height (in pixels), which does not necessarily equal display height

Can only be applied to video tracks and use not recommended, rely on DisplayHeight instead

TimeScale

Track's timescale as specified in server manifest

avc_level

Track's AVC level

Can only be applied to video tracks

avc_profile

Track's AVC profile (AVC_PROFILE_BASELINE, AVC_PROFILE_MAIN, AVC_PROFILE_HIGH)

Can only be applied to video tracks

DisplayWidth

Track's display width (in pixels)

Can only be applied to video tracks

SamplingRate

Track's audio sampling rate (in Hz)

Can only be applied to audio tracks

BitsPerSample

Track's resolution of the audio samples

Can only be applied to audio tracks

DisplayHeight

Track's display height (in pixels)

Can only be applied to video tracks

systemBitrate

Track's bitrate as specified in server manifest

systemLanguage

Track's language as specified in server manifest

FrameRate

Track's video frame rate (in frames per second)

Can only be applied to video tracks

ScanType

Track's video scan type ("progressive" or "interlaced")

Can only be applied to video tracks

Roles

Track's DASH role attribute

Contents of kind box where schemeId="urn:mpeg:dash:role:2011"

Dynamic Track selection example

  • When working with URIs including filter query parameters on the command-line, always put them in single quotes to avoid character escaping issues

  • To avoid character encoding issues while testing, use curl with its --data-urlencode option as it automatically takes care of URL safe character encoding (but don't forget to explicitly define --get because --data-urlencode defaults to HTTP POST)

  • To get direct feedback on whether a filter works or not and why, use mp4split -o stdout: locally with the server manifest as input and the virtual path to the client manifest as well as the filter query parameter appended (see examples below)

  • When testing playback in a browser, & must be escaped as %26

In this Tears of Steel Manifest example we have two audio (64 and 128 Kbps) and five video (400, 750, 1000, 1500 and 2200 Kbps) tracks.

Here are some example scenarios (and demo URLs) to show you what you can do with the Dynamic Track Selection feature.

Scenario 1: Select the lowest audio track and the lowest two video tracks.

Using curl with --data-urlencode (no specific character encoding necessary):

#!/bin/bash

curl --get 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest' \
  --data-urlencode 'filter=(type=="audio"&&systemBitrate<100000)||(type=="video"&&systemBitrate<800000)'

Or without --data-urlencode, using %26 to safely encode &:

#!/bin/bash

curl 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate<800000)'

Or using mp4split locally if you have downloaded the Tears of Steel vodpack (again, encoding & as %26):

#!/bin/bash

mp4split -o stdout: 'tears-of-steel-avc1.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate<800000)'

Scenario 2: Select the highest audio track and the two highest video tracks.

Again, using curl with --data-urlencode (no specific character encoding necessary):

#!/bin/bash

curl --get 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest' \
  --data-urlencode 'filter=(type=="audio"&&systemBitrate<100000)||(type=="video"&&systemBitrate>1300000)'

Or without --data-urlencode, using %26 to safely encode &:

#!/bin/bash

curl 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate>1300000)'

And using mp4split locally if you have downloaded the Tears of Steel vodpack (again, encoding & as %26):

#!/bin/bash

mp4split -o stdout: 'tears-of-steel-avc1.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate>1300000)'

Limiting bandwidth use example

The dynamic track selection feature can be used to cap the minimum/maximum bitrates returned in the manifest file as well.

Include all the video tracks with a bitrate higher than 800Kbps:

#!/bin/bash

curl -v 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate>800000)'

Similarly, include all the tracks where the bitrate is smaller than 800Kbps:

#!/bin/bash

curl -v 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate<800000)'

Dynamic Track selection capture example

Sometimes not all bitrates should be captured, but a selection. The following examples outline how to do that, combined with a start-end time.

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00-2015-12-08T15:21:00'

This captures only the 800k video bitrate (and the 64k audio bitrate).

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="17" />

which is about a minute (17 * 3.6 = 61.2, nearest GOP) and 14495879990000000 is the start time, Tue, 08 Dec 2015 15:19:59 GMT.

The following example shows how to capture a single audio and video bitrate from a certain point in time until the end (the current time):

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="141" />

which reads as from the time indicated until the end.

The URL used for the curl command in above examples can be used for capture as well:

#!/bin/bash

unified_capture -o test.ismv 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

Dynamic Track selection using count

New in version 1.7.27.

Sometimes tracks need to be selected by the presence of information of other tracks in the playlist. This can help in cases when the underlying manifest may not be known in advance, or when using the same filters for different manifests.

For example, there are two different manifests, which have either:

  • Video, EC-3 224 Kbps, AAC 192Kbps, AAC 64Kbps

  • Video, AAC 192Kbps, AAC 64Kbps

Suppose that you would need to write one filter that can achieve the following: Take all video tracks and, if there is a EC-3 track, take that, otherwise the highest AAC bitrate track.

With the new count operator it is possible to achieve this:

filter=(type=="video"||fourcc=="EC-3"||(count(fourcc=="EC-3")==0 && systembitrate==192000))

The expression count(fourcc=="EC-3")==0 checks whether if we count the number of tracks with EC-3 is exactly zero then evaluate && systembitrate==192000, select one AAC audio track. If there are EC-3 tracks, the && systembitrate==192000 will not be evaluated, skipping the other AAC audio track.

Usage: count(expr)

  • count: returns an integer value for the number of tracks in the playlist that match

  • expr: the expression to match all the tracks with

Rational number support

New in version 1.8.3.

It is possible to use rational numbers, for example with frame rates:

filter=(FrameRate == 30000/1001)

To filter to video tracks of 29.97 frames per second.

Dynamic Track selection using ScanType

New in version 1.10.2.

To be able to filter based on video scan type, there are two options, "interlaced" or "progressive".

For example with the video tracks:

  • 480x270p25

  • 640x360p25

  • 1024x576p25

  • 1280x720p50

  • 1920x1080i50

With the filter:

filter=(type=video&&ScanType=="progressive")

Will yield:

  • 480x270p25

  • 640x360p25

  • 1024x576p25

  • 1280x720p50

Using Rewrite Map for URL aliases

All options to configure a stream can be specified using query parameters in requests to Origin, except those related to DRM and archiving. Query parameters can also be used to filter tracks from a stream, using Using dynamic track selection. This can help you to dynamically cater a stream to different device requirements and environments.

Unfortunately, query parameters do not work well with CDNs, where caches typically either ignore anything after the first ? or proxy the request up-stream without caching. Preventing query parameters altogether also mitigates player issues caused by buggy implementations of RFC 3986.

When you do not want to include query parameters in the requests that a player makes, it is possible to set up pre-configured URL aliases that Apache rewrites to query parameters instead.

Such a workflow makes use of Rewrite Map in combination with --suppress_query_parameters and --presentation_name. It allows you to bundle device specific options by mapping custom profile names to settings in the form of query parameters. This section explains how to set this up, along with examples further down below.

The general idea is to expose playout URLs that include a symbolic virtual path between server and client manifest, like 'presentation.ism/virtual-path.mpd', and then set up rewrite rules that intercept and parse the virtual-path. The extracted fields are then looked up in pre-defined profile lists that map a name alias to a set of query parameters.

This has a number of benefits:

  1. Prevents exposing query parameters to player or CDN.

  2. Have one base server manifest for all content offered to various devices.

  3. Share media content (and/or HLS media playlist) URLs between client manifests. (To prevent unnecessary duplication and thus alleviate burden on CDN cache.)

  4. Generic and flexible filter mechanism to manage multiple server manifests.

It is possible to use file name prefixes and virtual directories as part of the virtual path that can enable a URL aliases scenario. These two options are described individually below. An example that combines both into one complete setup is detailed in Example: Multi-level profile aliases for HLS playout.

Specify aliases using file name prefixes and virtual directories

File name prefix based aliases

In the simplest case, query parameters affect only the selection of streams to be included in the client manifest, using Using dynamic track selection. To enable such a workflow with URL aliases, the virtual-path should be a plain (alphanumeric) file name prefix while the query parameters only contain rules for ?filter=, like ?filter=systemBitrate<500000.

This approach is relatively easy, because although the selection of streams in the client manifest differ per request URL, the content of each stream remains identical and can therefore be shared across the customized client manifests.

Virtual directory based aliases

To implement URL aliases that alter the underlying content, virtual directories should be used. This is useful when the aliases are mapped to query parameters that alter the actual content of a stream, for example by specifying a different segment length.

Normally, when exposing query parameters, this is not a problem. Because query parameters are propagated to all outbound URLs advertised by the client manifest, changes implied by the manifest are visible in the media request. This guarantees that distinct content is mapped to distinct URLs (e.g., for the same stream with different segment lengths).

To offer the same guarantee with URL aliases, the mapping should use a directory instead of a file name prefix in the virtual-path, when the alias corresponds to a query parameter that changes the content of a stream.

--suppress_query_parameters

New in version 1.10.7.

The option suppress_query_parameters instructs Origin to remove all query parameters from URLs (or URL templates) in client manifests.

Warning

If you make use of suppress_query_parameters, ensure that your rewrite rules are set up such that distinct content is still mapped to distinct URLs (e.g., for the same stream with different segment lengths) when query parameters are no longer propagated.

--presentation_name

New in version 1.10.8.

The option presentation_name overrides the name prefix used in URLs (or URL templates) in client manifests. Valid values are non-empty alpha numeric strings. By default presentation name is the basename of the server manifest (without .ism suffix).

This option is useful as a query parameter to propagate URL details from a HLS Master Playlist to its Media Playlists.

Example: Multi-level profile aliases for HLS playout

This example demonstrates how to set up Apache rewrite rules to combine pre-defined stream selection bundle (mobile, desktop, tv) with pre-defined format alterations (ts, fmp4) and optionally select a subclip timeline (intro, main).

It demonstrates HLS only, because this is the most intricate setup due to HLS's distinction between Master and Media Playlists. However, a similar configuration is possible for other playout formats. With that in mind, based on the different types of aliases described above, a request for a Master Playlist will take the following form:

presentation.ism/*format*/*bundle*@*timeline*.m3u8

Or, without timeline:

presentation.ism/*format*/*bundle*.m3u8

So we can mix and match:

presentation.ism/ts/mobile.m3u8
presentation.ism/fmp4/tablet.m3u8
presentation.ism/ts/tv@main.m3u8
presentation.ism/fmp4/tv@intro.m3u8
presentation.ism/fmp4/desktop.m3u8
...

The filter rewrite map filter-aliases.txt defines which of the available streams to make part of the Master Playlist:

mobile  filter=(type=="audio")||(systemBitrate==236000)
tablet  filter=(type=="audio")||(systemBitrate==370000)||(systemBitrate==571000)
tv      filter=(type=="audio")||(type=="video"%26%26systemBitrate>600000%26%26systemBitrate<1500000)
desktop filter=(type=="audio")||(type=="video"%26%26systemBitrate>600000)

Above, tracks are filtered based on name, bitrate, and type. For a full list of available filters see Using dynamic track selection. Note that %26%26 is the properly escaped form of && for the URL request.

The timeline rewrite map timeline-aliases.txt defines virtual subclips to include in the media playlist:

intro t=0-0:00:06.58
main  vbegin=0:00:06.59

The format alteration rewrite map alter-aliases.txt defines a profile for playout of Fragmented MP4 (fMP4) and another for Transport Streams (TS):

ts  hls_fmp4=false
fmp4 hls_fmp4=true&hls_client_manifest_version=7

The Apache mod_rewrite configuration looks like this:

RewriteEngine on

# Define maps to be used in RewriteRule
RewriteMap alter    "txt:/path/to/alter-aliases.txt"
RewriteMap timeline "txt:/path/to/timeline-aliases.txt"
RewriteMap filter   "txt:/path/to/filter-aliases.txt"

# HLS (Master Playlist)
RewriteRule "^(.*\.ism)/([^/]*)/(\w*)\.m3u8$"        "$1/.m3u8?${alter:$2}&${filter:$3}&suppress_query_parameters" [PT,L]
RewriteRule "^(.*\.ism)/([^/]*)/(\w*)@(\w*)\.m3u8$"  "$1/.m3u8?${alter:$2}&${filter:$3}&presentation_name=$4&suppress_query_parameters" [PT,L]

# HLS (Media Playlist)
RewriteRule "^(.*\.ism)/([^/]*)/(.*)-([^-]*)\.m3u8$" "$1/$4.m3u8?${alter:$2}&${timeline:$3}&suppress_query_parameters" [PT,L]

# HLS (Media segments)
RewriteRule "^(.*\.ism)/([^/]*)/(.*)\.ts$"           "$1/$3.ts?${alter:$2}" [PT,L]
RewriteRule "^(.*\.ism)/([^/]*)/(.*)\.aac$"          "$1/$3.aac?${alter:$2}" [PT,L]
RewriteRule "^(.*\.ism)/([^/]*)/?hls/(.*)\.webvtt$"  "$1/hls/$3.webvtt?${alter:$2}" [PT,L]
RewriteRule "^(.*\.ism)/([^/]*)/(.*)\.webvtt$"       "$1/$3.webvtt?${alter:$2}" [PT,L]
RewriteRule "^(.*\.ism)/([^/]*)/(.*hls/.*)\.m4s$"    "$1/$3.m4s?${alter:$2}" [PT,L]

The playlist rewrite rules extract the virtual directory name $2 as well as the file name prefix $3. If a key matching the directory name is found in (first column) of alter-aliases.txt, then the value (second column) is appended to the URL query, denoted by ${alter:$2}.

In case of the Master Playlist, the file name prefix is searched for in filter-aliases.txt and its value inserted into ${filter:$3}. The optional timeline string following @ is captured in $4 and forwarded to the prefix of any outbound URLs in the Media Playlist using the presentation_name query parameter.

The Media Playlist rewrite rule captures this customized presentation name in $3 and looks up a matching subclip in timeline-aliases.txt to replace ${timeline:3} in the URL query.

Note that as a consequence of the timeline propagation, there are now three distinct siblings of each Media Playlist: one with prefix intro-, one with main- and one starting with the presentation- default (full timeline). However, since they share the same base URL path they refer to a shared, overlapping set of media segments (thereby increasing caching efficiency).

In addition, the suppress_query_parameters option is added to all playlist requests to prevent propagation of the injected parameters. A passthrough [PT] flag is added for correct rewrite handling in mod_smooth_streaming.

Considerations when using RewriteMap for Smooth Streaming segment URLs

Because of the way Smooth uses virtual paths for segment URLs (e.g., adding QualityLevels(123000) to select quality 123000), a specific RewriteRule is needed to correctly handle segments URLs for Smooth when you use RewriteMap with a virtual subpath to map requests to a query parameter configuration.

For example, assuming you want to map requests with a 'smarttv' subpath to a RewriteMap 'filter', you should add a rule to explicitly intercept Smooth segment URLs to remove the 'smarttv' virtual path:

RewriteRule "^(.*\.ism)/([^/]*)/(QualityLevels.*)$" "$1/$3?${filter:$2}" [PT,QSA,L]

For a generic Smooth segment URL:

http://server/some/long/path/foobar.ism/smarttv/QualityLevels(123000)/Fragments(video=0)

The above RewriteRule regex would match the following groups:

​- $1 = http://server/some/long/path/foobar.ism - $2 = smarttv - $3 = QualityLevels(123000)/Fragments(video=0)

With the replacement string $1/$3?${filter:$2} making sure the subpath is removed from your segment URL (while still using the subpath to add query parameters using the RewriteMap you defined).

HTTP Dynamic Streaming (HDS) 'stream level' manifests

In HDS the configuration information about multi-bitrate streams has been divided into multiple levels: set-level F4M and stream-level F4M files. For HDS, the files are F4M manifest files.

--hds.multi_level

Set-level F4M files contain the URL to the stream-level manifest file and the bitrate information for each stream in a multi-bitrate set. For HDS, the set-level F4M file can also contain information about a DVR rolling window.

Name

Description

Set-Level Manifest

A manifest file that groups individual stream-level manifest files

Stream-Level Manifest

A manifest file describing a single <media> instance

 
#!/bin/bash

mp4split --hds.multi_level \
  --hds.client_manifest_version=2

Note

For HLS, there is a similar feature, called --variant_set.

Configuration and ordering of HLS playlists

Apple's HTTP Live Streaming (HLS) uses media groups and variant streams to sort the media contents of an ABR stream so that a client will know between which combinations of content it should switch. Media groups and stream variants are defined in a stream's Master Playlist.

A media group is the most basic sorting mechanism defined by HLS. It is used to group together different representations of content in media of an identical type (such as video or audio) with similar properties (like average bandwidth).

Variant streams represent the different varieties of a stream between which a client should switch for playout. Each variant is a unique combination of the stream's media, in which every type of media available in a stream is represented once:

  • As a group, if the particular type of media is sorted as such within the stream

  • As an individual track, if the track's media type is not sorted in a group(s)

Important

The variety of variant streams that is available in a stream is restricted by the way in which the different media in a stream are grouped.

How HLS groups and variants are defined

Definition of HLS media groups

The default behaviour is that tracks of the same media type are grouped together if their encoding is the same (i.e., if codecs are identical and, for audio and video, bitrate as well). This default configuration should suffice for almost all use cases.

In short, audio tracks that are similarly encoded but that represent different languages will be grouped, and this is true for subtitles in different languages as well, but video tracks that contain different bitrates will remain individual tracks.

Definition of HLS variant streams

By default, each variant will contain at least one representation of each media type that is represented in a stream. Thus, if a stream contains one group of subtitle tracks, this group will be part of each variant.

Other than that, the behaviour that defines the available variants is best understood starting from a stream that contains several audio groups and several video tracks (the logic is the same if the video tracks are also grouped, or if the audio tracks are not):

  1. Associate lowest bitrate audio group with lowest bitrate video track to generate a variant.

  2. Re-apply the first step, based on the remaining audio groups and video tracks.

  3. Continue this process until all audio groups and/or all video tracks are part of a variant stream.

  4. If either audio groups or video tracks remain, take the highest bitrate group/track of the other media type and associate it with each of the remaining groups/tracks.

Please find in the table below find an example that shows the result of the above process. In this example, two audio groups are associated with five video tracks to create five variant streams. Notice that the audio group with the lowest bitrate is part of only one variant stream:

Audio tracks

Audio groups

Video tracks

Variant streams

English 32kHz @ 64k

AAC-64

360p @ 256k

AAC-64 + 256k

Dutch 32kHz @ 64k

360p @ 512k

AAC-192 + 512k

Spanish, 32kHz @ 64k

720p @ 1024k

AAC-192 + 1024k

English, 48kHz @ 192k

AAC-192

1080p @ 2048k

AAC-192 + 2048k

Dutch, 48kHz @ 192k

1080p @ 4096k

AAC-192 + 4096k

Spanish, 48kHz @ 192k

If text tracks are also part of the stream, an additional step will define the final set of variant streams. Depending on whether the text tracks are grouped or not, each individual track or each group of tracks will be associated with each of the of the variants that were the result of the process described above.

Select variant stream that HLS clients should start playout with

It is possible to specify an initial bitrate that suits your client device best. With this option you can set the bitrate that the client starts playback with instead of leaving the initial selection to the client. This feature can be used when you want to skip the lowest bandwidth at start, for example.

It only influences the variant selection at the start of playback and does not influence playback after that. That is, it does not filter out any of the variants or media tracks that are available within a stream.

URL to the media presentation

Description

http://localhost/video/video.ism/video.m3u8?start_index=0

The client manifest start at the first indexed bitrate.

http://localhost/video/video.ism/video.m3u8?start_index=4

The client manifest starts at the fifth indexed bitrate.

If you would like to apply some kind of filter, you can use the above method in combination with Using dynamic track selection to actually filter out tracks before setting the start index for the client. This is done like so:

#!/bin/bash

curl -v 'https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel.ism/.m3u8?filter=(systemBitrate<1200000)&start_index=1'

In this example the two lowest video tracks are selected and the highest of them is set to be the initial bitrate.

Define default track when preparing content (track order)

New in version 1.7.17.

Apart from of using query parameters to specify the track that a player should start playback with, it is possible to define the default track for HLS playout when packaging the content for audio and subtitles (where the default is indicated with DEFAULT=YES in the Master Playlist).

To do so, it is necessary to order the tracks when packaging. To do this:

  • Create separate fMP4's that contain all of the stream's audio or subtitles tracks

  • When creating these fMP4's, make sure that the track that a player should start playback with is added on the command-line first

  • Create a server manifest based on all fMP4's like you normally would

  • For HLS playout, the audio and subtitles tracks that were first in order will be signaled as DEFAULT=YES in the HLS Master Playlist

For example, using the command-lines will ensure that audio track from tears-of-steel-aac-128k.mp4 and subtitles track from tears-of-steel-fr.mp4 are set to DEFAULT=YES:

#!/bin/bash

mp4split -o tos_avc1.ismv \
  tears-of-steel-avc1-1000k.mp4 \
  tears-of-steel-avc1-1500k.mp4 \
  tears-of-steel-avc1-750k.mp4 \
  tears-of-steel-avc1-400k.mp4

mp4split -o tos_aac-sorted.isma \
  tears-of-steel-aac-128k.mp4 \
  tears-of-steel-aac-64k.mp4

mp4split -o tos_subs-sorted.ismt \
  tears-of-steel-fr.mp4 \
  tears-of-steel-en.mp4

mp4split -o tears-of-steel-sorted.ism \
  tos_avc1.ismv \
  tos_aac-sorted.isma \
  tos_subs-sorted.ismt

Configure HLS variant streams

--variant_set

New in version 1.7.15.

Note

Before using this option, please read through Configuration and ordering of HLS playlists.

By default all the variants are listed (in no particular order) and it is for the player to decide which variant to play (although Apple does define the first variant that is listed as the 'default variant').

When the default behaviour does not give you the result you want, you can define a subset of tracks based on which the variants will be generated. If a track in such a subset is part of a group, the group will represent the track in the subset. This is true even if some tracks in the group were not part of the subset that was defined.

The syntax of the expression that defines a subset is the same as for Using dynamic track selection. You can add the expression using the variant_set-option when generating a server or HLS client manifest with mp4split.

It is possible to use the --variant_set-option more than once, to define several subsets. When doing so, the order in which you add the expressions will define the order of the variants in the Master Playlist. This makes the --variant_set-option useful for when you want to list a specific variant at the top of the Master Playlist.

If you only want to use the option to list a certain variant at the top of the Master Playlist, you should define two subsets that are jointly exhaustive, with the first subset containing a single track or group per media type so that the only variant that it will generate is the one that you want listed at the top.

For example, if we have a multi-bitrate stream like in the table above and we want to make sure that a variant stream with the 1024k video track is listed at the top, a subset that will generate this variant should be defined by the expression of the first --variant_set-argument in your mp4split command-line. The second subset that is defined should then cover all the other variants.

--variant_set='systemBitrate==1024000&&type=="video"||systemBitrate==64000&&type="audio"'
--variant_set='systemBitrate!=1024000||type!="video"'

Ordering HTTP Smooth Streaming (HSS), MPEG-DASH tracks

For Smooth Streaming and DASH, ordering the tracks in the manifest is not possible. Both specifications dictate that there should be no such ordering and that the players should make the appropriate choices. We follow the specification in that regard. However, if you want to make a certain selection of a server manifest's tracks available for playout, you can make use of a filter, as described in Using dynamic track selection.

IIS passthrough

Sometimes, pre-encoded VOD content is not encoded correctly. This can happen especially with older versions of Expression Encoder or Transform Manager.

When using IIS this might go unnoticed because IIS doesn't look at the actual content it's serving, it simply extracts a byte range and sends it along to the client. USP actually parses the sample data (for various reasons, for example converting to other formats, applying different DRM schemes).

When the content cannot be re-encoded, which is the preferred way to handle this problem as it is the content that is at fault, ISS passthrough mode may be used.

This will allow USP to still serve the content to VOD - as IIS does. However, as the content is no longer parsed transmuxing capabilities are lost: only ISS will play as it is 'passed through' - there is no transmuxing to HLS, HDS or DASH.

Apache

For Apache the directive looks like the following:

<Directory "/var/www/example">
  UspIssPassThrough On
  UspPreferStatic On
</Directory>

Please note that the UspPreferStatic is added as well. This is needed to ensure client manifests are not generated by USP (by reading the content), but instead are read from disk as well.

Both UspIssPassThrough and UspPreferStatic are used in the Directory context within the VirtualHost. Multiple directories can be used in a single virtual host, so this option can be used on a per directory basis.

Using Apache's Basic Authentication with IsmProxyPass

This module is compiled by default, it does not require an extra installation.

Server

It is mandatory to create a password file and add the user credentials. This credentials will be used on the file requests.

#!/bin/bash

sudo htpasswd -c /etc/apache2/.basicauth basicauthuser

Once you execute the command above, a password will be asked. Type your password and the file will be created in the Apache folder. You don't have to create the files in the Apache folder, it can be stored any folder but only the same server Apache had installed.

We will be securing our content in the directory /secure. In this tutorial we also use one of the Unified Streaming S3 Bucket for the IsmProxyPass target. Add the below configuration into your Virtual Host.

<Location "/secure">
  AuthType basic
  AuthName "private area"
  AuthUserFile    "/etc/apache2/.basicauth"
  Require            valid-user
</Location>

<Directory "/var/www/tears-of-steel/secure" >
  IsmProxyPass http://usp-s3-storage.s3-eu-central-1.amazonaws.com/
</Directory>

Note

The example configuration would only work if your Virtual Host's document root configured as "/var/www/tears-of-steel".

Restart your Apache server after all the changes are saved.

Client

A client now make a request only with using the authentication credentials to get the target files.

#!/bin/bash

sudo curl -v -u basicauthuser:yourpassword "http://your_server_address/secure/tears-of-steel/tears-of-steel.ism/.mpd

Notes

You don't have to create any folders into your Apache document root folder. The webserver translates the proxy request without a physical folder.

Apache basic authentication works with a standard Unified Origin installation.

This installation will only secure the folder stated in the configuration. Rest of the files can be requested from your web server without any authentication.

Using Apache's mod_auth_token

This module is not compiled by default, so you may have to install it first. Please see the mod_auth_token page for instructions.

Server

We will be securing our content in the directory /protected.

<Location "/protected/">
  UspSkipRewrite   off
  AuthTokenSecret  "secret"
  AuthTokenPrefix  "/protected/"
  AuthTokenTimeout 3600
</Location>

Note

From 1.10.26 onwards, Unified Origin does not rewrite incoming URLs anymore, by default. Effectively, the UspSkipRewrite directive is therefore on, if you never specify it.

Because mod_auth_token relies on this rewriting, you must explicitly enable it, by specifying UspSkipRewrite off for the location(s) that contain the mod_auth_token-related directives, as shown above.

Client

On the client we generate an MD5 hash for a specific URL.

#!/bin/bash

ISM_TIMESTAMP=$(php -r "print dechex(time());")
ISM_HASH=$(php -r "print md5('secret/alvin/alvin.ism$(ISM_TIMESTAMP)');")

The ISM_HASH hash is calculated over your secret password, the path to the video server manifest file and the current time.

The last thing to do is to add the hash and timestamp parameters to the URLs that you would normally provide to the player.

For HSS:

http://localhost/protected/$(ISM_HASH)/$(ISM_TIMESTAMP)/alvin/alvin.ism/manifest

For HLS:

http://localhost/protected/$(ISM_HASH)/$(ISM_TIMESTAMP)/alvin/alvin.ism/alvin.m3u8

For HDS:

http://localhost/protected/$(ISM_HASH)/$(ISM_TIMESTAMP)/alvin/alvin.ism/alvin.f4m

Notes

All the requests for a presentation (the playlists, the media playlists, the fragments) are secured, and thus all URLs must have the security hash and timestamp tagged on to their URL. Note that you do not have to do anything for this.

You only have to add the security attributes to the presentation manifest/playlist file. The webserver takes care of adding the security parameters to any URLs referenced in the playlists and media playlists.

Just make sure that you do not have static copies of the playlists (.m3u8 / .f4m / .ismc) stored on disk and let the webserver generate them dynamically.