HTTP Webserver Details

HTTP Response Headers

Changed in version 1.10.4.

USP distinguishes two cases: VOD and live.

Unified Origin only sets any response headers when they are applicable and there is no (feasible) way to set them with regular means (like mod_headers). For instance, only the Origin knows when a segment is going to be no longer available based on Live archiving settings.

But for VOD contents, where no intrinsic expiration constraints exist, we do not insert default values allowing Cache-Control and/or Expires headers to be customized in webserver configuration or left unspecified (depending on CDN or player requirements).

Last-Modified

For VOD the Last-Modified header is always set to the last modified time of the server manifest file (.ism). Note that this applies to the manifest/playlist as well as any segments.

For Live the Last-Modified time is set to:

  • The modification time of the server manifest (.isml) for client manifest, Master Playlist and initialization segments

  • The time of ingest of the segment with the highest timestamp for media playlists

  • The time of ingest of the segment (for segments)

ETag

Changed in version 1.10.4.

Both VOD and live have an ETag value that is a hash of URL's response content.

Depending on the request, the hashed data comprises the manifests or generated MP4 data, but excludes the (static) elementary stream (media data). The ETag changes only when the payload changes (for example as a result real-time changes, dynamic filtering or playout control) and If-None-Match yields 304 only if nothing changed.

Expires and Cache-Control: max-age="..."

Changed in version 1.10.4.

Changed in version 1.10.16.

For VOD no Cache-Control or Expires headers are set.

However to ensure no caching issues when a live publishing point becomes VOD (or is used for catch up) it maybe necessary to manually set a Cache-Control header.

This can be achieved using the following line in your apache virtual host configuration file (note that this will set max-age=2 for all Master Playlists requested from a Live publishing point, even when no End of Stream signal has been sent).

<LocationMatch "^.*\.isml\/.*(\.mpd|\.m3u8|Manifest)$">
  Header setifempty Cache-Control "max-age=2"
</LocationMatch>

For Live the Cache-Control and Expires are set to:

  • Dynamic client manifests like HLS Media Playlists expire when a new segment is expected to be ingested (the sum of the last segments's ingest time and its duration)

  • Segments near the Live edge expire when new revisions become available as a result of timing information in successive 'lookahead_fragments' (applicable to Smooth output only)

  • For static client manifests like a HLS Master Playlist no Cache-Control or Expires headers are set

Note

As indicated above, no Cache-Control or Expires headers are set for media segments in a Live stream (with the exception of the latest few segments of a Smooth Live stream). Instead, a Sunset header is added, which indicates when a segment will no longer be available.

Sunset

New in version 1.10.17.

The Sunset header applies to Live streaming scenarios only. It is added to media segments and it signals when a media segment will no longer be available.

A publishing point with constrained archive length (see: --archive_length) will periodically purge old archive segments to free up disk storage. The advertised sunset time for a requested media segment reflects the anticipated time at which the archive segment containing the media will be removed (assuming the encoder continues to push contents).

When a stream is closed (because a End of Stream was sent to each track), no Sunset header is written anymore (because media segments will no longer be removed from Live archive).

For cache configuration purposes, you may wish to add additional scripting (apache mod_lua) or custom logic to use the Sunset header to configure Expires and Cache-Control max-age=X headers. The Expires value can be set to the value of the Sunset and the max-age value could be expressed as an integer (seconds) equating to the difference between time "now" and the Expires value. All of this is best configured on your shield cache or CDN.

More info on Sunset header: RFC 8594.

Usp-Cache-Control: max-age-ms="..."

New in version 1.14.3.

Usp-Cache-Control is inserted in the same media objects as Cache-Control. However, Usp-Cache-Control provides a millisecond precision compared to a second precision of Cache-Control. This response header provides the key max-age-ms= and an integer value in milliseconds. Usp-Cache-Control response header can improve cache efficiency at the distribution layer (e.g., CDN, cache layer, etc.) when lower latencies are paramount.

To support this feature it is required to parse the Usp-Cache-Control response header and set the value to max-age-ms as the object's TTL in your cache logic.

CMSD-Static: at=...,ept=...

New in version 1.14.2.

For VOD no CMSD-Static: at=...,ept=.. headers are set.

The CMSD-Static: at=...,ept=.. header applies to Live streaming scenarios only. It is added to mp4 media segments and it signals the Availability Time (at) and the Earliest Presentation Time (ept). Each value is described as follows:

  • Availability Time (at): it is the timestamp when the media fragment was written to disk at Origin in milliseconds since epoch time (1970-01-01-00:00:00 UTC).

  • Earliest Presentation Time (ept): it is the timestamp of the media_time from the ProducerReferenceTimeBox with (in ISO/IEC 14496-12 ) of the first media sample in milliseconds since epoch time (1970-01-01-00:00:00 UTC).

More info on CMSD-Static header: CMSD cta-5006.

Custom HTTP Status codes (Apache)

For live presentations a request for a media fragment may result in one of the following errors (depending on the time of the media and the current state of the available DVR window):

Error

Description

Fragment Not Found

The requested fragment is (no longer) available.

Fragment Not Available

The requested fragment is newer than the last fragment available in the DVR window.

You can customize the following status codes returned by the webserver module for live presentations. Note that this is customized per playout format.

Configuration key

HTTP status code

Fmp4IssFragmentNotFound

404

Fmp4IssFragmentNotAvailable

412

Fmp4HdsFragmentNotFound

404

Fmp4HdsFragmentNotAvailable

503

Fmp4HlsFragmentNotFound

404

Fmp4HlsFragmentNotAvailable

404

Fmp4MpdFragmentNotFound

404

Fmp4MpdFragmentNotAvailable

404

The customization in Apache can be specified on a per-directory (including sub directories) basis using Directory sections, or matching URI locations, using Location/LocationMatch sections.

An example Apache configuration (using the default status codes) using a Directory section:

<Directory "/var/www/live/usp_test/video/errors_rewrite">
  Fmp4IssFragmentNotFound 404
  Fmp4IssFragmentNotAvailable 412
</Directory>

An example Apache configuration (using the default status codes) using a LocationMatch section:

# Customize status codes for DASH fragments
<LocationMatch "\.dash$">
  Fmp4MpdFragmentNotFound 404
  Fmp4MpdFragmentNotAvailable 404
</LocationMatch>

Note

Customization of status codes for virtual (i.e. non-existing) directories, such as are usually used for dynamic manifests (with IsmProxyPass, see Dynamic Manifests) should only be used in Location or LocationMatch sections.

This is because Apache can only apply directives in Directory sections to paths which actually exist on the server's file system. This limitation does not apply to Location sections.

Starting and stopping Apache

There are a couple of ways to stop, start or restart Apache, which are described here.

Please note that a restart (signal: HUP) will stop all connections, incoming and outgoing.

A 'graceful restart' (signal: USR1) will not, so outgoing streams continue. However, a Live ingest (the encoder POSTing to the publishing point) will stop, even with a graceful restart. There is no way to reset Apache and keep the ingest intact.

Logrotate

Please note that logrotate will reset all connections, as it does a restart of Apache. This might cause a problem with a Live ingest.

There are two ways to address this:

  • turn off log rotation for the ingest (the encoder POST will only log when it ends, so the logging is minimal anyway)

  • use rotatelogs.