HLS Manifest structure question

[giladna]

Hi,

We have HLS clear manifest with the following structure - variats bitrates and keyframes bitrates are not matching excactly.

The issue is that Exo Player demo app exposes all the bitrates that are in the streams but actually the bitrates related to the keyframe are different then the ones for the video tracks so we get all of them but actually when trying to switch to the keyframe bitrates which s not in the varients the video gets stuck

could you suggest if this is correct behaviour or this is stream issue?

we can also see during the playback that there are jumps in the frames.

#EXTM3U
#EXT-X-VERSION:4

Created with Broadpeak BkS350 Origin Packager (version=1.11.3-24483)

AUDIO groups

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aach-128",LANGUAGE="Original",NAME="Original",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-aach-128",LANGUAGE="cs",NAME="Czech",AUTOSELECT=YES,CHANNELS="2",URI="1094-audio_129102_cze=128000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"

variants

#EXT-X-STREAM-INF:BANDWIDTH=500000,AVERAGE-BANDWIDTH=454000,CODECS="mp4a.40.5,avc1.4D4029",RESOLUTION=320x180,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aach-128",CLOSED-CAPTIONS=NONE
1094-audio_129100_mul=128000-video=300000.m3u8?deviceType=301&subscriptionType=20998&locality=IL
#EXT-X-STREAM-INF:BANDWIDTH=1083000,AVERAGE-BANDWIDTH=984000,CODECS="mp4a.40.5,avc1.4D4029",RESOLUTION=640x360,FRAME-RATE=25,VIDEO-RANGE=SDR,AUDIO="audio-aach-128",CLOSED-CAPTIONS=NONE
1094-audio_129100_mul=128000-video=800000.m3u8?deviceType=301&subscriptionType=20998&locality=IL
#EXT-X-STREAM-INF:BANDWIDTH=2598000,AVERAGE-BANDWIDTH=2362000,CODECS="mp4a.40.5,avc1.640029",RESOLUTION=960x540,FRAME-RATE=50,VIDEO-RANGE=SDR,AUDIO="audio-aach-128",CLOSED-CAPTIONS=NONE
1094-audio_129100_mul=128000-video=2100000.m3u8?deviceType=301&subscriptionType=20998&locality=IL
#EXT-X-STREAM-INF:BANDWIDTH=4114000,AVERAGE-BANDWIDTH=3740000,CODECS="mp4a.40.5,avc1.640029",RESOLUTION=1280x720,FRAME-RATE=50,VIDEO-RANGE=SDR,AUDIO="audio-aach-128",CLOSED-CAPTIONS=NONE
1094-audio_129100_mul=128000-video=3400000.m3u8?deviceType=301&subscriptionType=20998&locality=IL
#EXT-X-STREAM-INF:BANDWIDTH=6446000,AVERAGE-BANDWIDTH=5860000,CODECS="mp4a.40.5,avc1.64002A",RESOLUTION=1920x1080,FRAME-RATE=50,VIDEO-RANGE=SDR,AUDIO="audio-aach-128",CLOSED-CAPTIONS=NONE
1094-audio_129100_mul=128000-video=5400000.m3u8?deviceType=301&subscriptionType=20998&locality=IL

keyframes

#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=44000,CODECS="avc1.4D4029",RESOLUTION=320x180,VIDEO-RANGE=SDR,URI="keyframes/1094-video=300000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=117000,CODECS="avc1.4D4029",RESOLUTION=640x360,VIDEO-RANGE=SDR,URI="keyframes/1094-video=800000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=307000,CODECS="avc1.640029",RESOLUTION=960x540,VIDEO-RANGE=SDR,URI="keyframes/1094-video=2100000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=496000,CODECS="avc1.640029",RESOLUTION=1280x720,VIDEO-RANGE=SDR,URI="keyframes/1094-video=3400000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=788000,CODECS="avc1.64002A",RESOLUTION=1920x1080,VIDEO-RANGE=SDR,URI="keyframes/1094-video=5400000.m3u8?deviceType=301&subscriptionType=20998&locality=IL"

[giladna]

I was wondering if some how the end result is related to that?
https://github.com/google/ExoPlayer/blob/release-v2/docs/troubleshooting.md#why-do-some-mpeg-ts-files-fail-to-play

thanks

[tonihei]

The issue is that Exo Player demo app exposes all the bitrates that are in the streams but actually the bitrates related to the keyframe are different then the ones for the video tracks so we get all of them

This part is expected. ExoPlayer parses these #EXT-X-I-FRAME-STREAM-INF tags and adds them as formats to the same TrackGroup as the normal variants. See some reasoning for this in #474 (comment).
You can distinguish the tracks by looking at the role flags. See #8313 where a similar question has been asked.

It's worth noting that ExoPlayer's DefaultTrackSelector does not select these tracks under any circumstances unless you explicitly set a TrackSelectionOverride to select this track.

but actually when trying to switch to the keyframe bitrates which s not in the varients the video gets stuck

This part is not expected. But we also haven't tested playing I-frame-only tracks in detail and I wouldn't be surprised if there are a few problems left. #8938 sounds potentially related because playback also got stuck when playing these tracks with audio. Could you try disabling audio to see if it makes a difference?

[giladna]

@tonihei I was checking the same media with disabling the audio and it will play images one after another when I select sine bitrate from the keyframes

so first I have filtered out all those tracks so I will expose for track selection only the valid tracks with audio.

now I was wondering how do you think the trick play stand alone will be managed I guess ABR will not be working on these tracks right?

SO say I have to use these tracks for Fast Forward Feature I will have to disable audio and use the trickplay bitrate that most fits the current video track bitrate being used and set player speed as per user request

is it correct approach?

[tonihei]

so I will expose for track selection only the valid tracks with audio

This happens automatically in DefaultTrackSelector unless you specifically set a TrackSelectionOverride including the trick play tracks.

I guess ABR will not be working on these tracks right

If you include them in a selection, ABR should theoretically work, but I wonder what you expect to happen? A user with a very slow network would suddenly see a trick-play tracks with a small number of key frames and not the expected smooth playback. One could argue that seeing a few frames is better than seeing no frame, but just wanted to make you aware that this might not be expected.

SO say I have to use these tracks for Fast Forward Feature I will have to disable audio and use the trickplay bitrate that most fits the current video track bitrate being used and set player speed as per user request

This should work. Note that proper trick play support is quite complicated as you'd want to adjust more properties of the player. See #3752 (comment) for some further thoughts (especially points 4 onwards, which also apply to HLS)

[tonihei]

I was checking the same media with disabling the audio and it will play images one after another

Based on this, I'll close this issue as a duplicate of #8938.