• AnyStream is having some DRM issues currently, Netflix is not available in HD for the time being.
    Situations like this will always happen with AnyStream: streaming providers are continuously improving their countermeasures while we try to catch up, it's an ongoing cat-and-mouse game. Please be patient and don't flood our support or forum with requests, we are working on it 24/7 to get it resolved. Thank you.

Resolved Process MP4 and MKV Files with mov_text or ASS Subs for Usage With Plex

jbrisbin

Well-Known Member
Thread Starter
Joined
Nov 30, 2007
Messages
393
Likes
50
Background:
I heavily post-process all my downloaded content. First by re-encoding to HEVC at a high quality level into MKV containers, then by converting the ASS subtitles to SRT (for Plex compatibility).

Note: While Plex nominally support ASS subtitles, it appears to seek to them by scanning the media file data from the beginning of the file to locate them. When you are distant from the server, even on a 100Mb connection this is suboptimal and usually times out. This problem might no longer exist, but I have not seen any release notes indicating it has been fixed.
You can test this by starting a movie file with ASS subs present but not enabled, then skip in 30 minutes and enable an ASS sub track. Resume playback. It should resume instantly if the issue is fixed. If not, playback will probably timeout before all the data to that point can be read over a slow connection to find the next sub.

Update:
A new version of the script is now available. This script has a new name 'subs.sh', some optimizations and new functionality though using it is very similar to the original version.
The updated version will convert the subs in both .mkv and, now, .mp4 files but it always produces a .mkv output file.
Functionally, the handling of .mkv files is the same.
In the case of .mp4 files, it depends on whether the file contains text based subs. If so, they are extracted and converted to SRT form and reinserted into the output .mkv file. If no text subs are found in the .mp4, it is simply remuxed into the output .mkv file with no conversion.
In either case any .mp4 files below or in the current directory will be converted. This may not be desirable, in which case you may want to use the original mkvsrt.sh command, instead.
I have been using the newer subs.sh version for about 6 months and it has served me well. I continue to re-encode the AVC output of AP and NF, but now directly convert the HEVC output of HM.​

This post is about the process of converting those subtitles and the script I use to do it.

As part of the second stage, I set the forced flag on anything that has the word 'forced' in the name of the subtitle. If you have multiple 'forced' subtitles in your video file, the result is undefined and you probably won't like it. The default flag is carried to the converted subtitle that had the default flag before conversion.

All the subtitle processing is done in a Cygwin/bash script that recursively processes directories of video MKV files. Thus, it can (to the extent you trust it) process an entire Plex library directory with a single invocation (though it may take quite a long time).

This has been thoroughly tested on a single Windows 10 system. This is another way of saying that a large number of really obvious problems would probably appear immediately if anyone were to run it on a different Windows or Linux system.
It does backup the video it changes (so you need an equal amount of space to any files with ASS subs that are processed). It removes the ASS subs after it adds the SRT equivalents, so nothing is processed two or more times. Each file is sanity checked before the backup/replace operation occurs. All tools used are free (primarily mkvtoolnix and ffmpeg). It works equally well on TV episodes and movies. It puts out too much console information for anybody but me.

Installation:
Two packages are required to be installed to use the script:
mkvtoolnix
ffmpeg
For standalone linux, use the usual distribution package manager to install them.
For Cygwin, which does not offer a package manager other that the setup program, you can use that setup as described here:
https://blueskyworkshop.com/blog/2015/09/10-cygwin-packages.html
Or add the third party package manager apt-cyg as described here:
https://techdirectarchive.com/2020/...packages-from-the-command-line-using-apt-cyg/
Unfortunately, that approach requires wget or lynx which may not be installed, forcing you back to the setup method to get them. Probably best to use setup and be done with the operation.


The attached script is called 'mkvsrt.sh'. It should be placed in a location on the default path of your CygWin setup (Windows) or bash shell (linux). Under CygWin, '/usr/local/bin/' appears to be such a place. Once placed there, make it executable by running the command (assuming that you put it in /usr/local/bin/):
chmod +x /usr/local/bin/mkvsrt.sh
If you are using the newer subs.sh version, the command would be:
chmod +x /usr/local/bin/subs.sh

Before you begin:
Your video files may be in MKV (.mkv) containers or in the MP4 (.mp4) containers typically produced by AS

If you have MP4 containers (.mp4 files) you must use the new version of the script: subs.sh.
If you have MKV containers, you can use either script. The newer, subs.sh will perform somewhat better when more than one ASS sub is embedded because it will only make one pass over the file to extract all subtitles, instead of one pass each.


Usage:
open a shell (either Cygwin from windows or bash from linux).
cd to the directory at the top of the file collection you want to process. Initially, pick a directory with one or a few files at or below the directory you choose. If the script causes your computer to catch fire, this will limit the damage.

To convert both MP4 (.mp4) and MKV (,mkv) files, execute the new script by typing its name:
subs.sh

To covert only MKV (.mkv) files, execute the original script by typing its name:
mkvsrt.sh

Once you press return, it will begin searching for mp4 and mkv files. It will check each one for ASS subtitles and, if present, begin processing the file. Files without ASS subtitles will be ignored. This allows you to restart the process without much loss if it stops or you stop it for some reason (running out of disk space, for example).
As each file is inspected, and potentially processed, quite a bit of text will be output to the shell window. If something goes wrong, this is the text you will need to capture to report an issue.

Last thing the script does is to save the original video file in a subdirectory with a name like this: SubConvBackup 2021-05-30 00.39.14
Inside that directory, the script places a .plexignore file which makes the backup (old) files invisble to Plex. Without this each movie processed would appear as a duplicate. The date and time in the backup directory name are set each time you launch the script. This will prevent one backup from overwriting another.

When and if you are satisfied that the video has be faithfully converted, delete any backup directories to recover the space.

Good luck. Proceed cautiously.
 

Attachments

  • mkvsrt.zip
    3.4 KB · Views: 38
  • subs.zip
    4.6 KB · Views: 20
Last edited:
I'm sure your work will be appreciated, but according to Plex ASS subtitles are supported anyway
 
If the ones generated by AS worked under Roku, Amazon Fire, or Shield TV Plex clients, I would be more impressed.
 
As I said in another post. I download the srt file and then use mkvtoolnix to combine the SRT and the mp4 to a mkv with no remux, just a container change. No change in quality. Works fine with plex using my nVidia shield.
 
I mentioned this in some other threads...
The subtitles generated by AnyStream are not ASS , because ASS (or SSA) is not supported in MP4 files while TTXT (or tx3g) is, you can check this with MediaInfo
You are correct. The mux to MKV, which I find necessary to avoid glitches and hangs in remote video playback, deals with that major issue, but requires additional processing to include tx3g subs.
My apologies, I was writing from memory about things I investigated some time back.
The real issue is with the output of Handbrake which performs an implicit (non-negotiable with developers) conversion from tx3g to ASS. MKVToolnix just ignores the subs and does not copy them at all if tx3g. This is also non-negotiable with the developer.
Since I have made the informed decision to transcode to HEVC (you may differ, but it doesn't matter to me), Handbrake is almost the only choice. Except for the ASS subs that perform poorly, if at all with Plex.
The issue with plex is that if you turn on ASS subs after starting, it appears to scan up to the current point in the film to find the sub, if you are patient enough, it will display one subtitle, then continue on.
If you are remote, 'patient enough' results in a timeout. If you enable the sub before starting play back, it works until you skip, then they are gone or you are hung (and not well).
These may be purely Handbrake/Plex artifacts, but I can't be sure.

These considerations make the script I have published of far less general interest than I had thought.

I will update it so that it can handle .mp4 input.

The result should be that the output will avoid the problems with hangs and stalls during simple non-sub playback (because of remux to mkv), support automatic forced subs where present (adding forced flag where appropriate), and predictable and consistent behavior locally and remotely (because if Plex supports ASS, it supports SRT much better). tx3g is academic with an mkv container.

Hopefully, it won't take long.
 
I guess I'm confused as to why use ASS when SRT works fine? Like I said, remux (thanks for the correction, was typing too fast previously) to MVK with the mp4 and external SRT using mkvtoolnix. Then you can encode to x265 if you want using Handbrake and it should work perfectly. I've done the x265 conversion myself but personally didn't think the time it took was worth it to compress a compressed stream download. But it did work.
 
...

The real issue is with the output of Handbrake which performs an implicit (non-negotiable with developers) conversion from tx3g to ASS. MKVToolnix just ignores the subs and does not copy them at all if tx3g. This is also non-negotiable with the developer.
Since I have made the informed decision to transcode to HEVC (you may differ, but it doesn't matter to me), Handbrake is almost the only choice. Except for the ASS subs that perform poorly, if at all with Plex.

I would strongly recommend StaxRip here ...
It's not as easy to use as Handbrake, but it can convert almost anything to anything.
Once you set your output template, you can feed whole folders to the program and all output is the same.
For me its audio conversion to 640kb AC3 for compatibility, subtitles to SRT (no issues at all with Plex) and if needed recoding to HEVC with NVENC (got my Turing graphics card the mining mania let the prices go through the roof) else I just remux the videostream
 
So I've run this and it successfully works for tracks without a title, but when a subtitle track has text, such as 'SDH' (without the quotes), there is a parsing exception in the json subParams file:

Error: The JSON option file './subParams.json' contains an error: [json.exception.parse_error.101] parse error at li
olumn 9: syntax error while parsing value - invalid string: control character U+000D (CR) must be escaped to \u000D
last read: '"0:SDH<U+000D>'.
Failed to merge subtitle to tmp-out-srt.mkv!

For me, my subParams.json file looks like this:

[

"./tmp-in-srt.mkv",
"--language",
"0:eng",
"--default-track",
"0:0",
"--track-name",
"0:SDH
",
"./out.srt",
"--output",
"./tmp-out-srt.mkv"
]

The error with with the "--track-name","0:SDH
"

It appears that the newline character isn't being picked up as part of the regex checking in the script:

# trim spaces fore and aft
trackName=`echo $trackName | sed -e 's/^[[:space:]]*//'`
 
Last edited:
The problem, as the error message indicates in the usual obscure style, is that the 0:SDH entry is followed by a carriage return (CR). The error in your subParams.json file is that on the "0:SDH line the closing quote appears on the following line with the comma. That line should have been
"0:SDH",
instead.
Control characters such as the carriage return embedded in a string are, it seems, illegal in well formed json.
 
The problem, as the error message indicates in the usual obscure style, is that the 0:SDH entry is followed by a carriage return (CR). The error in your subParams.json file is that on the "0:SDH line the closing quote appears on the following line with the comma. That line should have been
"0:SDH",
instead.
Control characters such as the carriage return embedded in a string are, it seems, illegal in well formed json.

Yes, I was just making you aware that there is an error scenario not handled in the script. I'm making a change on my end to adjust for this scenario.

Edit:
For anyone else coming along with the same issue, the following modification can be made on line 208:
trackName=`echo $trackName | sed -e 's/^[[:space:]]*//' | sed 's/\r$//'`

Edit2:
On another note, thank you for the work you did OP. Saves me the time of having to write something like this myself to process my library. Really appreciated!
 
Last edited:
Ah, I see.
The subParams file is generated from the original subtitle embedded as you noted. Do you know what program generated the video file with what is likely a bad name for the trackname in the subtitle stream? It should not have had a CR there, either.
 
Ah, I see.
The subParams file is generated from the original subtitle embedded as you noted. Do you know what program generated the video file with what is likely a bad name for the trackname in the subtitle stream? It should not have had a CR there, either.
I'm not sure - I got the file from an external source. My main reason for even finding this post today is because last night I was streaming a tv show on Plex on my Nvidia shield and noticed it was transcoding 1080p h.265 to 1080p h.264 due to the .ASS subtitles being turned on.

I could have addressed the issue manually using mkvtoolnix to edit the subtitle track name, but that obviously defeats the purpose of it all being automated :)
 
Thanks for the report and the explanation. I am sure I can manage to trim all whitespace from the ends rather than just from the front end.
 
Last edited:
Kyle S, I have PM'ed you with an updated version that still works with the files that I have. Can you verify that it successfully addresses the file you have with CR embedded in the track name?
Thanks for your help.
 
Here is a new version of the mkvsrt.sh script. It has a minor change to remove trailing whitespace from the trackname field. The absence of this fix caused the issue brought up by Kyle S.
While the change has not broken the case where it worked before, I don't have a test case for the whitespace characters that follow the trackname, so I cannot be sure that it really fixes that relatively rare issue.
The new version is attached to this post and will become the first post version if no issues crop up.
 

Attachments

  • mkvsrt29Sep21.zip
    3.4 KB · Views: 20
Just as a follow-up, I've been using the latest script and have not been running into any issues like I did with the initial version. Both your updated version as well as the modified version of the original that I made are working flawlessly for all of my anime files using .ass that I am converting to use .srt
 
Just as a follow-up, I've been using the latest script and have not been running into any issues like I did with the initial version. Both your updated version as well as the modified version of the original that I made are working flawlessly for all of my anime files using .ass that I am converting to use .srt

Thanks for checking and updating us.
 
Back
Top