Program Usage

Unified Capture is designed to be used in tandem with Unified Origin. The recommended streaming format to capture from is MPEG-DASH, as this is the most versatile. The MPD should use a $Time$ -based SegmentTemplate (i.e. an explicit SegmentTimeline).

We recommend:

Using Unified Capture to only capture media from Unified Origin
Playing out the captured content through Unified Origin and/or Unified Remix only
Using only matching software versions of Unified Capture and Unified Origin
Using a Using a dedicated capture server manifest

Where a dedicated capture server manifest is not used certain options MUST be negated by use of query parameters for example '?fixed_gop=0'.

However, for various reasons one may want to use Unified Capture to capture streams that do not originate from Unified Origin directly. For instance, when a third-party service is relied upon for Dynamic Ad Replacement or Insertion (which may generate a Multi-Period DASH manifest), and the goal is to capture the stream including the replaced or inserted ads.

For unsupported use cases like these your mileage may vary, but you will increase your chances of success by ensuring the inserted ads are equal to the original stream with respect to:

Number of tracks
Order of tracks
Technical quality (encoding)

Command line options 

Unified Capture is a command-line driven program. Its generic format is:

unified_capture -o output_file [options] <input-url>

Supported input

The <input-url> needs to be a fully qualified URL that points to a client manifest dynamically generated by Unified Origin. The URL can be used to either fetch the manifest via HTTP or to address it on the local filesystem — the same options are supported.

Note that the part of the URL specifying the client manifest is a virtual path so that even when addressing the stream through the local filesystem only the server manifest exist on disk while the client manifest does not (because it will be dynamically generated while capturing).

Looking at the example below, the first part of the URL is 'real'. It points to the server manifest, which is stored on disk (/var/www/live/channel01/channel01.isml). The /.mpd?t=... part that follows however, is 'virtual', because it points to the (DASH) client manifest, which isn't stored on disk but generated on the fly.

The part of the path that addresses the client manifest, comes the time range that you want to capture. The format for this time range is the ISO 8601 date-time format ("2013-03-31T12:34:56.000" and it is specified using t as a query parameter:

#!/bin/bash

unified_capture -o video.ismv \
  "file:///var/www/live/channel01/channel01.isml/.mpd?t=2013-03-31T12:00:00.000-2013-03-31T12:30:00.000"

And when using a request via HTTP, the command looks very similar:

#!/bin/bash

unified_capture -o video.ismv \
  "http://live.unified-streaming.com/channel01/channel01.isml/.mpd?t=2013-03-31T12:00:00.000-2013-03-31T12:30:00.000"

Options for Capturing 

--license-key

The license key

--copy_ts

Copy the timestamp from the source (default is to start at 0).

--frame_accurate

Enable frame accurate mode.

--remix

The --remix option creates a frame accurate subclip with original timestamps (so using --copy-ts is not necessary) and no transcoding. These special subclips are intended to be used in a Unified Remix - nPVR workflow where they are called archive segments. Because archive segments generally do not start with a sync sample, they cannot be used as-is for regular media processing and playback.

The original media can be restored seamlessly using a concatenation of multiple consecutive captured archive segments using Unified Remix. For further information please see Unified Remix - nPVR.

Note

When capturing for a Unified Remix - nPVR workflow it is important that the client manifest produced by Unified Origin contains all samples in the sub-clip range. Some server-side playout options, like --time_shift may prevent this from happening. Like with all use cases for Unified Capture we therefore recommend using a Using a dedicated capture server manifest.

--key

The 128 bits Key ID (KID) and 128 bits Content Encryption Key (CEK) are passed with the --key option to Decrypt using --key command-line option the stream.

--decrypt_cpix=...

Specify description key(s) through CPIX document.

--daemon

Use this option to run Unified Capture in daemon mode. This will also increase the number of potential retries from 3 (normal mode) to 30.

-v

Verbosity level.

Frame accurate 

By default, the begin and end times of a captured clip are aligned to video key frames. When the --frame-accurate option is used, Unified Capture transcodes the first and last fragments of a clip to allow for sample accurate timing.

Note

The --frame_accurate option requires a working media processing setup. Please see the Media processing architecture overview, and Plugin selection: the transcoders file for further details.

Also note that when using Unified Origin or Unified Packager to generate progressive MP4's from frame accurate captured content, only advanced players will be able to play it.

Decrypt using `--key` command-line option 

When the presentation is encrypted, you can decrypt and store an unencrypted stream.

To do so the KID and CEK used for encryption/DRM must be passed to capture using the --key option, separating KID and CEK by a colon, e.g. --key=KID:CEK.

The KID identifier uniquely identifies the content. The CEK is the Content Encryption Key.

In case the encryption/DRM does not use a KID the CEK may be passed as --key=:CEK, omitting the KID.

Example:

#!/bin/bash

KID=000102030405060708090a0b0c0d0e0f
CEK=000102030405060708090a0b0c0d0e0f

unified_capture -o clip1.ismv \
  --key=${KID}:${CEK} \
  "http://example.com/video1/video1.ism/Manifest"

Decrypt using CPIX with `--decrypt_cpix`

It is also possible to specify a CPIX document containing one or more <ContentKey> (s) that have matching KID(s) to the content.

#!/bin/bash

unified_capture -o clip1.ismv \
  --decrypt_cpix=name.cpix \
  "http://example.com/video1/video1.ism/Manifest"

This option is available for both VOD and Live streams.

Decrypting clips can for instance be useful for QoS purposes (to monitor normally encrypted/DRM'ed streams) or to maintain a clear archive for catch-up services.

Clips produced by Unified Capture may be post-processed with the Unified Packager to create other formats.

Filtering out specific tracks when capturing a stream may be applied make a selection of tracks or just one track to be captured.

Filtering out specific tracks when capturing a stream 

When capturing all bitrates of a stream is not needed, see the Dynamic Track selection capture example for an example on how to filter the captured stream using an expression which tracks should be included (and which not).

Exception handling 

To some extent Unified Capture mimics the behaviour of a typical player. If an HTTP request fails, it can retry the request to mitigate a temporary problem in the network, CDN or origin.

Occasionally, a request will fail simply because there was a problem with the input media. While some players are capable of "driving around" such a gap, this is a severe problem with the presentation.

Retry

If an HTTP request for a segment does not result in status 200 OK, Unified Capture will assume this is caused by a transient problem. After a 4 second grace period, it may re-attempt once or twice, depending on the HTTP status code it received from Unified Origin.

Maximum number of retries	HTTP status code
2	404 (Not found)
	410 (Gone)
	416 (Range not satisfiable)
1	403 (Forbidden)
	408 (Request timeout)
	503 (Service unavailable)

This behavior is based on DVB recommendations for player resilience (section 1.10.8 in DVB-DASH specification (ETSI TS 103 285)) and may change in the future.

Missing Segments

New in version 1.11.9: Create gaps for missing fragments

When a segment that is advertised in the client manifest cannot be downloaded, then Unified Capture will attempt to continue (after retrying). To avoid misaligned media, Unified Capture will synthesize an empty fragment based on the timing information in the manifest and insert this gap segment into the fMP4 before continuing with the next segment.

Another reason why Unified Capture might insert gap segments is when the client manifest signals a gap discontinuity. Gaps in the client manifest are typically caused by interruption of the encoder or network problems at the ingest side of the origin.

When the capture process completes on a file where gaps have been added (i.e. it reaches the end of the stream, clip end or is interrupted by the user) it will report the error and signal the problem though a non-0 exit status.

The resulting fMP4 is obviously erroneous, but even so, may sometimes be useful. For instance if one bitrate has problems, all other tracks could still be salvaged.

Attention

We recommend monitoring exit status of Unified Capture (as well as the terminal output) to avoid unintentionally hiding or suppressing problems. Under normal circumstances Unified Capture will always return the customary 0 exit code.

Using a dedicated capture server manifest 

for a capture manifest, use the same archive_length, archive_segment_length and archiving setting as your regular publishing point, and specify database_path if relevant, set mpd.segment_template to time, potentially use a higher value for mpd.minimum_fragment_length and specify timed_metadata if timed metadata is involved and you want to capture it. nothing else

While it is possible to capture straight from an Origin endpoint used to deliver streams to viewers, it is recommended and often more convenient to set up an additional (internal) endpoint specifically for Unified Capture. To avoid abuse, the web server should be configured to block any access to the internal endpoint, except the for the nPVR capture process.

A typical server manifest geared at public distribution may enable features and tweaks that complicate the capture process. Additionally, later stages of the streaming workflow (e.g., manifest manipulation) can complicate this further.

To some extent it is possible to mitigate those from the capturing process. For example, by providing a decryption key to remove DRM or adding a time_shift=0 query parameter to the request URL to prevent incorrect timestamps. But this is brittle: a change to the playout may necessitate additional mitigation of the capture configuration.

To sidestep these problems, it is often easier to create an additional server manifest. This is essentially a copy of the original pubpoint, but with as little options as possible. Both server manifests should reside in the same directory and --database_path=... should point to the same .db3 file. For live capture, the only mandatory option is --mpd.segment_template=time. It is also possible to omit / remove streams that you choose not to archive.

For example:

`primary-pubpoint.isml`

<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
    <meta name="clientManifestRelativePath" content="primary-pubpoint.ismc" />
    <meta name="lookahead_fragments" content="2" />
    <meta name="dvr_window_length" content="28800" />
    <meta name="archive_segment_length" content="60" />
    <meta name="archiving" content="true" />
    <meta name="archive_length" content="172800" />
    <meta name="restart_on_encoder_reconnect" content="true" />
    <meta name="hls_minimum_fragment_length" content="48/25" />
    <meta name="mpd_availability_start_time" content="1970-01-01T00:00:12Z" />
    <meta name="mpd_minimum_update_period" content="2" />
    <meta name="mpd_suggested_presentation_delay" content="14" />
    <meta name="iss_client_manifest_version" content="22" />
    <meta name="hls_client_manifest_version" content="5" />
    <meta name="hls_no_audio_only" content="true" />
    <meta name="hls_no_multiplex" content="true" />
    <meta name="mpd_min_buffer_time" content="4" />
    <meta name="iss_playout" content="playready" />
    <meta name="hls_playout" content="aes" />
    <meta name="hds_playout" content="faxs" />
    <paramGroup id="hds">
      <meta name="key" content=":A8FD3449772FA3DD2F1BCE74764A8B46" />
    </paramGroup>
    <paramGroup id="hls">
      <meta name="key" content=":A8FD3449772FA3DD2F1BCE74764A8B46" />
      <meta name="license_server_url" content="http://example.com/hls.key" />
    </paramGroup>
    <paramGroup id="iss">
      <meta name="key" content="030201000504070608090A0B0C0D0E0F:A8FD3449772FA3DD2F1BCE74764A8B46" />
    </paramGroup>
    <meta name="timed_metadata" content="true" />
  </head>
  <body>
    <switch>
      <audio src="stream0.ismv" systemBitrate="32000" systemLanguage="eng" />
      <audio src="stream0.ismv" systemBitrate="48000" systemLanguage="eng" />
      <audio src="stream0.ismv" systemBitrate="64000" systemLanguage="eng" />
      <meta src="stream1.ismv" systemBitrate="0" />
      <video src="stream2.ismv" systemBitrate="100000" />
      <video src="stream3.ismv" systemBitrate="112000" />
      <video src="stream4.ismv" systemBitrate="130000" />
    </switch>
  </body>
</smil>

`remix-npvr.isml`

<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
    <meta name="database_path" content="/path/to/primary-pubpoint.db3" />
    <meta name="archive_segment_length" content="60" />
    <meta name="archiving" content="true" />
    <meta name="iss_playout" content="false" />
    <meta name="mpd_playout" content="true" />
    <meta name="hds_playout" content="false" />
    <meta name="hls_playout" content="false" />
    <meta name="mpd_segment_template" content="time" />
    <meta name="timed_metadata" content="true" />
  </head>
  <body>
    <switch>
      <audio src="stream0.ismv" systemBitrate="64000" systemLanguage="eng" />
      <meta src="stream1.ismv" systemBitrate="0" />
      <video src="stream4.ismv" systemBitrate="130000" />
    </switch>
  </body>
</smil>

If network latency between origin and capture incurs high overhead, it may help to increase --mpd.minimum_fragment_length. As always, take care that the value reflects a multiple of your GOP length.

For example: if your media is encoded with 1.92s GOPs and your public playout uses segments of this length, the capture process can double the segment length to 3.84s by adding --mpd.minimum_fragment_length=96/25. This will halve the number of segments that Unified Capture needs to request to capture the clip and therefore halve the cumulative round-trip time, which may be a significant decrease when capturing across remote regions.