Support S_TEXT/ASS subtitles in MKV

[personshelldon]

Hello, I'm working on a relatively simple TV and for this I chose ExoPlayer. Now there is a question about reading subtitles from video files. To initialize the player, I use the following code:

DefaultBandwidthMeter bandwidthMeter = new DefaultBandwidthMeter(mainHandler,null);
        DataSource dataSource = new DefaultUriDataSource(getContext(), bandwidthMeter, userAgent);
        sampleSource = new ExtractorSampleSource(Uri.parse(url), dataSource, allocator, 200 * 64 * 1024);
        MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(getContext(),sampleSource,
                MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,
                1500,
                mainHandler,
                new MediaCodecVideoTrackRenderer.EventListener() {
                    @Override
                    public void onDroppedFrames(int i, long l) {
                    }

                    @Override
                    public void onVideoSizeChanged(int videoWidth, int videoHeight, int i2, float v) {
                        currentVideoParameters = new VideoParameters(videoWidth,videoHeight,v);
                        invalidateVideoSize();
                    }

                    @Override
                    public void onDrawnToSurface(Surface surface) {

                    }

                    @Override
                    public void onDecoderInitializationError(MediaCodecTrackRenderer.DecoderInitializationException e) {

                    }

                    @Override
                    public void onCryptoError(MediaCodec.CryptoException e) {

                    }

                    @Override
                    public void onDecoderInitialized(String s, long l, long l1) {

                    }
                },5);
        audioRenderer = new MediaCodecAudioTrackRenderer(sampleSource);
// this line was added to read subtitles
        TrackRenderer textRenderer = new TextTrackRenderer(sampleSource,this,mainHandler.getLooper());
        mMediaPlayer.prepare(videoRenderer, audioRenderer, textRenderer);
        mMediaPlayer.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surfaceView.getHolder().getSurface());

Video and audio is playing fine, but the subtitles are not working. If I call getTrackCount(TEXT_RENDER) it gives me 0, but in video file there are three tracks of subtitles. I am not good in any codecs or in the principles of playing, I basically application programmer;
For testing I'm using this video file: http://www.ex.ua/get/201122663

How I can force to work subtitles in my project? Thanks and sorry for my English.

[IanDBird]

What's the format of the subtitles embedded in the MKV? I tried to download the file via the link you provided but just go:

Sorry, you can't download files...

[personshelldon]

Sorry, you're right. This site work only in Ukraine. I uploaded this file to Google Drive:
https://drive.google.com/file/d/0B2j8OTRVFCUEYUNMT3Y0TzVscXM/view?usp=sharing

This is our test video.

[ojw28]

• For what it's worth, the original link also works in the US (but not in the UK) :).
• Subtitles are S_TEXT/ASS, which we do not currently support. We currently support S_TEXT/UTF8 only for MKV. Let's use this as a feature request to add support, however note that we will not be prioritising it internally, for the time being.

[IanDBird]

ASS subtitles can be incredibly complex to render. I wonder if it might be possible/feasible to somehow leverage libass (https://github.com/libass/libass/) as a native extension to render to a separate canvas to be composited on top of the video...

[ojw28]

Looking at this, it doesn't look particularly complicated? What are the specific things you're concerned about complexity wise?

[IanDBird]

The spec that you're linking too only includes very basic examples. In practice, ASS subtitles are used to construct very complicated subtitle renderings. Here's an example such a subtitle, which is a couple of years old, http://puu.sh/kTknq/eab4c5c9a1.png.

People who create ASS based subtitles spend an incredible amount of time producing very detailed subtitles. I think you might be under-estimating how difficult it actually is to properly support them. Here is another example, where all the yellow "ribbons" are added via the subtitle: http://puu.sh/kTkxI/a692a60803.png.

[ojw28]

Heh, they look non-trivial :). Thanks for the info!

[IanDBird]

Here's another crazy example, where mask vectors have been used: http://fat.gfycat.com/MemorableGoldenAphid.gif :)

[jeoliva]

Wow, never saw that kind of subtitle... Definitively something much more
complex than typical subtitles we are used to see.

El jue., 22 oct. 2015 12:51, Ian Bird notifications@github.com escribió:

Here's another crazy example, where mask vectors have been used:
http://fat.gfycat.com/MemorableGoldenAphid.gif :)

—
Reply to this email directly or view it on GitHub
#889 (comment).

[ojw28]

Simple SSA/ASS subtitles (not yet including styling) are now supported in dev-v2.

[ojw28]

Closing this as simple subtitles are now supported, and it's unclear whether there's demand for lots of styling support for this subtitle type. Please file a new enhancement specifically about styling if it's a requirement. Thanks!