HLS Playlist Delta Updates

HLS Playlist Delta Updates are a way to optimize live streams by reducing the amount of data transferred for each playlist request.

During a live stream a client is constantly reloading the media playlist to discover new segments. But when presented with a large DVR window (during a long running event), the playlist can become large (thousands of lines) even using gzip.

Playlists become longer also when trying to optimize for low latency, which in general implies adopting shorter segment lengths, thus requiring more segments to cover the same time window.

Therefore reloading a large playlist over and over takes time and bandwidth. This can sometimes cause the player to switch down to a lower quality tier, for instance if it was on a bad network.

Even when such playlists can be correctly transferred by the network, low-end devices with limited resources can still struggle to parse thousands and thousands of line frequently, impacting the quality of experience.

Playlist Delta Updates are a new way to reload an HLS playlist, where the server sends only the most recent changes to the client. The client then combines these updates with the version of playlist it already has to reconstruct what's already on the server.

How Playlist Delta Updates work

This a high-level overview on how Delta Updates work, based on a short extract from the HLS specification, aimed to provide the minimal necessary information to understand how to enable Delta Updates in Unified Origin. For a complete specification, please refer to HLS RFC8216.

Playlist Delta Updates features can be divided in "version 1" and "version 2".

Version 1

Delta Updates "version 1" allows skipping media segments URLs only in playlists.

A server advertises support for skipping media segments using a EXT-X-SERVER-CONTROL tag that includes:

  • CAN-SKIP-UNTIL: to advertise the capability of the server to produce playlists that skip older media segments.

When a Server receives a request for a Playlist containing the CAN- SKIP-UNTIL attribute but no EXT-X-ENDLIST tag, and the requested URI contains an _HLS_skip directive whose value is YES (or v2, see next section), it MUST respond with a Playlist Delta Update, which is a version of the playlist where media segments older than the Skip Boundary are replaced by an EXT-X-SKIP tag.

Version 2

Delta Updates "version 2" includes all functionalities of version 1, plus the chance to also skip EXT-X-DATERANGE tags.

A server advertises support for also skipping #EXT-X-DATERANGE tags using a EXT-X-SERVER-CONTROL tag that includes:

  • CAN-SKIP-DATERANGES: to advertise the capability of the server to produce playlists that skip older EXT-X-DATERANGE tags.

When the _HLS_skip directive has a value of v2, the Playlist Delta Update additionally MUST NOT contain EXT-X-DATERANGE tags that were added to the Playlist more than CAN-SKIP-UNTIL seconds before the Playlist request.

Notice that, despite what the server advertises, it is completely up to the player to request either "versions" of playlists, by either using an ?_HLS_skip=YES or an ?_HLS_skip=v2 query parameter.

How to enable Delta Updates in Unified Origin

Delta Updates are disabled by default in Unified Origin. They can be turned on by using the option

  • --hls.delta_updates

when using Unified Packager to create a live server manifest (.isml).

If you have existing server manifests that you don't want to repackage, the same can be accomplished by adding the line

  • <meta name="hls_delta_updates" content="true" />

in the head section of an existing server manifest.

Here are examples of playlists generated by Unified Origin when Delta Updates are enabled.

When players do not use any ?_HLS_skip query parameter:

 1#EXTM3U
 2#EXT-X-VERSION:4
 3## Created with Unified Streaming Platform  (version=1.14.6-31018)
 4#EXT-X-MEDIA-SEQUENCE:901824643
 5#EXT-X-INDEPENDENT-SEGMENTS
 6#EXT-X-TARGETDURATION:3
 7#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=18,CAN-SKIP-DATERANGES=YES
 8#USP-X-TIMESTAMP-MAP:MPEGTS=5295604128,LOCAL=2024-11-13T13:08:32.640000Z
 9#EXT-X-PROGRAM-DATE-TIME:2024-11-13T13:08:32.640000Z
10#EXTINF:1.92, no desc
11scte35-audio_eng=128000-video=1000000-901824643.ts
12#EXTINF:1.92, no desc
13scte35-audio_eng=128000-video=1000000-901824644.ts
14#EXTINF:1.92, no desc
15scte35-audio_eng=128000-video=1000000-901824645.ts
16#EXTINF:1.92, no desc
17scte35-audio_eng=128000-video=1000000-901824646.ts
18#EXTINF:1.92, no desc
19scte35-audio_eng=128000-video=1000000-901824647.ts
20#EXTINF:1.92, no desc
21scte35-audio_eng=128000-video=1000000-901824648.ts
22#EXTINF:1.92, no desc
23scte35-audio_eng=128000-video=1000000-901824649.ts
24#EXTINF:1.92, no desc
25scte35-audio_eng=128000-video=1000000-901824650.ts
26#EXTINF:1.92, no desc
27scte35-audio_eng=128000-video=1000000-901824651.ts
28## splice_insert(auto_return)
29#EXT-X-DATERANGE:ID="14314677-1731503329",START-DATE="2024-11-13T13:08:49.920000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB57FFFFE0034BC00C00000000000E4612424
30#EXT-X-CUE-OUT:38.4
31#EXT-X-PROGRAM-DATE-TIME:2024-11-13T13:08:49.920000Z
32#EXTINF:1.92, no desc
33scte35-audio_eng=128000-video=1000000-901824652.ts
34#EXTINF:1.92, no desc
35scte35-audio_eng=128000-video=1000000-901824653.ts
36.
37.
38.
39[Long list of segments and #EXT-X-DATERANGES adding many lines to the playlist]

When players use an ?_HLS_skip=YES request:

 1#EXTM3U
 2#EXT-X-VERSION:9
 3## Created with Unified Streaming Platform  (version=1.14.6-31018)
 4#EXT-X-MEDIA-SEQUENCE:901824643
 5#EXT-X-INDEPENDENT-SEGMENTS
 6#EXT-X-TARGETDURATION:3
 7#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=18,CAN-SKIP-DATERANGES=YES
 8#EXT-X-SKIP:SKIPPED-SEGMENTS=303
 9#USP-X-TIMESTAMP-MAP:MPEGTS=5295604128,LOCAL=2024-11-13T13:08:32.640000Z
10## splice_insert(auto_return)
11#EXT-X-DATERANGE:ID="14314677-1731503329",START-DATE="2024-11-13T13:08:49.920000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB57FFFFE0034BC00C00000000000E4612424
12## splice_insert(auto_return)
13#EXT-X-DATERANGE:ID="14314678-1731503450",START-DATE="2024-11-13T13:10:50.880000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB67FFFFE0034BC00C00000000000E4612424
14## splice_insert(auto_return)
15#EXT-X-DATERANGE:ID="14314679-1731503571",START-DATE="2024-11-13T13:12:51.840000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB77FFFFE0034BC00C00000000000E4612424
16## splice_insert(auto_return)
17#EXT-X-DATERANGE:ID="14314680-1731503692",START-DATE="2024-11-13T13:14:52.800000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB87FFFFE0034BC00C00000000000E4612424
18## splice_insert(auto_return)
19#EXT-X-DATERANGE:ID="14314681-1731503813",START-DATE="2024-11-13T13:16:53.760000Z",PLANNED-DURATION=38.4,SCTE35-OUT=0xFC302000000000000000FFF00F0500DA6CB97FFFFE0034BC00C00000000000E4612424
20#EXTINF:1.92, no desc
21scte35-audio_eng=128000-video=1000000-901824946.ts
22#EXTINF:1.92, no desc
23scte35-audio_eng=128000-video=1000000-901824947.ts
24#EXTINF:1.92, no desc
25scte35-audio_eng=128000-video=1000000-901824948.ts
26#EXTINF:1.92, no desc
27scte35-audio_eng=128000-video=1000000-901824949.ts
28#EXTINF:1.92, no desc
29scte35-audio_eng=128000-video=1000000-901824950.ts
30#EXTINF:1.92, no desc
31scte35-audio_eng=128000-video=1000000-901824951.ts
32#EXTINF:1.92, no desc
33scte35-audio_eng=128000-video=1000000-901824952.ts
34#EXTINF:1.92, no desc
35scte35-audio_eng=128000-video=1000000-901824953.ts
36#EXTINF:1.92, no desc
37scte35-audio_eng=128000-video=1000000-901824954.ts
38#EXTINF:1.92, no desc
39scte35-audio_eng=128000-video=1000000-901824955.ts

Notice the much shorter list of media segments, the updated #EXT-X-VERSION value (version 9) and the presence of the #EXT-X-SKIP tag. Notice also the presence of all the #EXT-X-DATERANGE tags that were present in the original playlist. The total amount of lines have been reduced drastically.

When players use an ?_HLS_skip=v2 request:

 1#EXTM3U
 2#EXT-X-VERSION:10
 3## Created with Unified Streaming Platform  (version=1.14.6-31018)
 4#EXT-X-MEDIA-SEQUENCE:901824643
 5#EXT-X-INDEPENDENT-SEGMENTS
 6#EXT-X-TARGETDURATION:3
 7#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=18,CAN-SKIP-DATERANGES=YES
 8#EXT-X-SKIP:SKIPPED-SEGMENTS=303,RECENTLY-REMOVED-DATERANGES=""
 9#USP-X-TIMESTAMP-MAP:MPEGTS=5295604128,LOCAL=2024-11-13T13:08:32.640000Z
10#EXTINF:1.92, no desc
11scte35-audio_eng=128000-video=1000000-901824946.ts
12#EXTINF:1.92, no desc
13scte35-audio_eng=128000-video=1000000-901824947.ts
14#EXTINF:1.92, no desc
15scte35-audio_eng=128000-video=1000000-901824948.ts
16#EXTINF:1.92, no desc
17scte35-audio_eng=128000-video=1000000-901824949.ts
18#EXTINF:1.92, no desc
19scte35-audio_eng=128000-video=1000000-901824950.ts
20#EXTINF:1.92, no desc
21scte35-audio_eng=128000-video=1000000-901824951.ts
22#EXTINF:1.92, no desc
23scte35-audio_eng=128000-video=1000000-901824952.ts
24#EXTINF:1.92, no desc
25scte35-audio_eng=128000-video=1000000-901824953.ts
26#EXTINF:1.92, no desc
27scte35-audio_eng=128000-video=1000000-901824954.ts
28#EXTINF:1.92, no desc
29scte35-audio_eng=128000-video=1000000-901824955.ts

Notice in this case how, on top of the skipped media segments, related #EXT-X-DATERANGE tags are removed as well. #EXT-X-VERSION value in this case is 10.

Note

The implementation of #EXT-X-DATERANGE skipping is willingly left partial, as it can be noticed by the empty RECENTLY-REMOVED-DATERANGES field. The specification in fact does not clarify how a packager is supposed to calculate the list of tags to be included in this field. This partial implementation has proven to be compatible with all players that have been tested and represents in our opinion the best implementation achievable at this stage. Should you encounter any issue or should you need this field filled, we would love to hear your use case and discuss a Feature Request: please reach out at support@unified-streaming.com.

Warning

Some CDN providers suggest to prevent caching for Delta Updates playlist (e.g. see Cloudfront recommendations on HLS Delta Updates ). If you do so, it is recommended to keep an eye on the performance analytics of your Origin as this can increase the load on the packager.

Also, if your stream is part of a Server Side Ad Insertion workflow, please ensure your Ad Insertion provider is compatible with Delta Update before enabling them.