Apple 16x9 basic stream: I-FRAME only track playing only without audio - with audio, the player is stuck

[Avetri]

I got spare not robust video tracks on HLS. (Tracks without resolution information in the logcat.)

Android:
7

ExoPlayer:
2.12.2 and 2.13.2

Streams (in HLS part of demo player media list):

Apple 4x3 basic stream
Apple 16x9 basic stream

Logcat:

2021-05-13 19:09:40.322 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:     Group:0, adaptive_supported=YES [
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:0, id=0, mimeType=video/avc, bitrate=263851, codecs=avc1.4d400d, res=416x234, supported=YES
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:1, id=1, mimeType=video/avc, bitrate=28451, codecs=avc1.4d400d, supported=YES
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:2, id=2, mimeType=video/avc, bitrate=577610, codecs=avc1.4d401e, res=640x360, supported=YES
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:3, id=3, mimeType=video/avc, bitrate=181534, codecs=avc1.4d401e, supported=YES
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:4, id=4, mimeType=video/avc, bitrate=915905, codecs=avc1.4d401f, res=960x540, supported=YES
2021-05-13 19:09:40.323 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:5, id=5, mimeType=video/avc, bitrate=297056, codecs=avc1.4d401f, supported=YES
2021-05-13 19:09:40.324 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:6, id=6, mimeType=video/avc, bitrate=1030138, codecs=avc1.4d401f, res=1280x720, supported=YES
2021-05-13 19:09:40.324 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:7, id=7, mimeType=video/avc, bitrate=339492, codecs=avc1.4d401f, supported=YES
2021-05-13 19:09:40.324 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [X] Track:8, id=8, mimeType=video/avc, bitrate=1924009, codecs=avc1.4d401f, res=1920x1080, supported=YES
2021-05-13 19:09:40.324 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:       [ ] Track:9, id=9, mimeType=video/avc, bitrate=669554, codecs=avc1.4d401f, supported=YES
2021-05-13 19:09:40.324 17805-17805/com.google.android.exoplayer2.demo D/EventLogger:     ]

[Avetri]

There is no such thing in ExoPlayer 2.5.4.

[christosts]

The tracks you are seeing are EXT-X-IFRAME-ONLY tracks, which were supported since 2.12. I can verify this is what is being reported when playing the Apple 16x9 basic stream, but I don't see the EXT-X-IFRAME-ONLY tracks on Apple 4x3 basic stream.

Are you seeing these on both streams?

[Avetri]

@christosts Thank You.

You are right. There are no spare (I-frame) video tracks in "Apple 4x3 basic stream". It was my fault. I am sorry.

[Avetri]

@christosts what kind of support do the EXT-X-IFRAME-ONLY tracks have? I got an exception when try to play them:

2021-05-14 22:43:32.817 12495-14546/com.google.android.exoplayer2.demo E/LoadTask: Unexpected exception loading stream
      java.lang.IllegalStateException
        at com.google.android.exoplayer2.util.Assertions.checkState(Assertions.java:86)
        at com.google.android.exoplayer2.util.TimestampAdjuster.setFirstSampleTimestampUs(TimestampAdjuster.java:60)
        at com.google.android.exoplayer2.source.hls.HlsMediaChunk.loadMedia(HlsMediaChunk.java:402)
        at com.google.android.exoplayer2.source.hls.HlsMediaChunk.load(HlsMediaChunk.java:355)
        at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:415)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
        at java.lang.Thread.run(Thread.java:761)
2021-05-14 22:43:32.819 12495-12495/com.google.android.exoplayer2.demo E/EventLogger: internalError [eventTime=81.89, mediaPos=78.98, window=0, period=0, loadError
      com.google.android.exoplayer2.upstream.Loader$UnexpectedLoaderException: Unexpected IllegalStateException: null
        at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:436)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
        at java.lang.Thread.run(Thread.java:761)
     Caused by: java.lang.IllegalStateException
        at com.google.android.exoplayer2.util.Assertions.checkState(Assertions.java:86)
        at com.google.android.exoplayer2.util.TimestampAdjuster.setFirstSampleTimestampUs(TimestampAdjuster.java:60)
        at com.google.android.exoplayer2.source.hls.HlsMediaChunk.loadMedia(HlsMediaChunk.java:402)
        at com.google.android.exoplayer2.source.hls.HlsMediaChunk.load(HlsMediaChunk.java:355)
        at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:415)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607) 
        at java.lang.Thread.run(Thread.java:761) 
    ]

[christosts]

They should be playable. There was a bug fixed in 2.13.3 so please use 2.13.3+ (2.14.0 was just released). From the demo app, you can watch "Apple master playlist advanced" (TS or FMP4) or "Apple media playlist (TS)" and select to view an i-frame-only track.

I can see that with "Apple 16x9 basic stream" on 2.14 playback does not start and the player is in buffering state, I will update this issue what's happening with this specific stream.

[christosts]

The player cannot play the I-FRAME track on the "Apple 16x9 basic stream" asset with audio, but it can play the I-FRAME without audio.

The asset has audio and video muxed on the same ts files and the I-FRAME track points on the same ts file (although it points to only the I-frames). That confuses the player because it is downloading a file that the player things contains both audio and video but cannot find the audio. In that case, the player remains stuck. If you select to play only video (and no audio), the player will play the I-FRAME track correctly.

Because this is not a regular use-case, we are marking this as low-priority.

[Avetri]

Ok. Thanks.

[stevemayhew]

@christosts I thought i-Frame only was excluded by this logic?

We are using a TrackSelectionParameter, setPreferredTrickPlayOnly() the idea is this would include/remove trick-play tracks and disable audio, ID3 and embedded captions. These additional renderers defeat the purpose of using trick-play video tracks to quickly locate a seek position and are never a valid use-case with trick-play (perhaps one might make a case for embedded CC tracks if they are included in the i-Frame only sample stream)

I'm assuming the user did this with the track overrides in the demo UI, which can lead to many un-supported use cases. If you can assign it to me and I can mark it fixed when the track-selection parameters are added, otherwise I would just close it as "works as designed"