[-BUG- FIXED in 1.1.1.0] [1.1.0.0-1.1.0.3] [All providers] Non-deterministic track truncation

Discussion in 'AnyStream' started by 0x0x0x0x0, Apr 2, 2021.

  1. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    While experimenting with "things I know worked before, let's make sure they work now before I fully commit" methodology, it appears that there's some wacky non-determinism in track truncation on N (not tested others) and it's non-deterministic as seems to be changing between closing and staring the application:

    Code:
    !V title.en.1.0.9.1.mp4 |v @ 1920x1080|=47:49.0617 |a E-AC-3 @ 768000 (en)|=47:51.0424 (-1.807 s)
    !V title.en.1.1.0.0.run.1.mp4 |v @ 1920x1080|=46:49.0015 |a E-AC-3 @ 768000 (en)|=47:38.0848 (-49.833 s)
    !V title.en.1.1.0.0.run.2.mp4 |v @ 1920x1080|=46:49.0015 |a E-AC-3 @ 768000 (en)|=47:46.0880 (-57.865 s)
    
    !V title.ru.1.0.9.1.mp4 |v @ 1920x1080|=47:49.0617 |a E-AC-3 @ 640000 (ru)|=47:51.0584 (-1.967 s)
    !V title.ru.1.1.0.0.run.1.mp4 |v @ 1920x1080|=46:49.0015 |a E-AC-3 @ 640000 (ru)|=47:46.0880 (-57.865 s)
    !V title.ru.1.1.0.0.run.2.mp4 |v @ 1920x1080|=46:49.0015 |a E-AC-3 @ 640000 (ru)|=47:46.0880 (-57.865 s)
    
    Upto 2 seconds of track truncation seems to be normal for N, but not 45+ seconds... I shan't mention the verboten release, sic loquitur pro se.
     

    Attached Files:

  2. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    Also happening in D, going off by playing the titles, there should be no discrepancies, at least not this early on. Actually, having checked with D's web player, it looks like both get truncated but by different amounts, even track durations look suspiciously identical in sets:

    Code:
    !V Title_A_S01E03_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:40.0000 (-6.648 s)  [Actual duration: 23:32, according to D's web player]
    !A Title_A_S01E04_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:32.0000 (1.352 s) [Actual duration: 23:27]
    !V Title_A_S01E05_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:40.0000 (-6.648 s) [Actual duration: 23.29]
    !V Title_A_S01E06_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:48.0000 (-14.648 s) [Actual duration: 23:21]
    !A Title_A_S01E07_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:32.0000 (1.352 s) [Actual duration: 23:30]
    !V Title_A_S01E08_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:40.0000 (-6.648 s) [Actual duration: 23:24]
    !V Title_A_S01E09_.mp4 |v @ 640x480|=22:41.0360 |a AAC @ 128000 (en)|=22:48.0000 (-6.64 s) [ ... and so on ... ]
    !V Title_A_S01E10_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:40.0000 (-6.648 s)
    !A Title_B_S01E01_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:24.0000 (1.344 s)
    !V Title_B_S01E02_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !V Title_B_S01E03_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !V Title_B_S01E04_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !A Title_B_S01E05_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:24.0000 (9.352 s)
    !V Title_B_S01E06_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !A Title_B_S01E07_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:24.0000 (1.344 s)
    !V Title_B_S01E08_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !V Title_B_S01E09_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !V Title_B_S01E10_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !A Title_B_S01E11_.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:24.0000 (1.344 s)
    

    Cross-ref for similar bug with A with logs: https://forum.redfox.bz/threads/subtitles-not-showing-up-on-my-anystream-files.80695/#post-536075
     

    Attached Files:

    Last edited: Apr 3, 2021
  3. Xander

    Xander Well-Known Member

    would it be possible for you to share this script again and maybe a step by step for dummies (like me) how to use it in windows? dident get it to work last time. pretty please with a cherry on top? :)
     
  4. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    You don't need the whole script (it does other checking), but this oneliner in powershell would do it:
    Code:
    $thresh=1000; gci PATH_TO_MP4s -Filter '*.mp4' | %{ $name=$_.FullName; C:\MediaInfo\MediaInfo.exe "--Inform=Video;%Duration% `nAudio;%Duration%" $name | %{ $l=$_.Split(' '); $d=$l[0] - $l[1];if([math]::abs($d) -gt $thresh){ $d/=1000; "!" + (&{if($d -lt 0){"V "}else{"A "}}) + $name + " |v|=" + ("{0:hh\:mm\:ss\,fff}" -f [timespan]::FromMilliseconds($l[0])) + " |a|=" + ("{0:hh\:mm\:ss\,fff}" -f [timespan]::FromMilliseconds($l[1])) + " ($d s)"};}}
    
    $thresh is the discrepancy threshold in milliseconds (1000 is good for A, but N needs about 2000, and even then some titles would be off despite being properly complete, don't know what it ought to be for D just yet)

    You'll need MediaInfo (the CLI version), grab the latest at
    Code:
    https://mediaarea.net/download/binary/mediainfo/21.03/MediaInfo_CLI_21.03_Windows_x64.zip
    
    Replace
    Code:
    C:\MediaInfo\MediaInfo.exe
    
    with the path to wherever you install MediaInfo, escaping spaces etc (or just install it in C:\MediaInfo ;) ), and
    Code:
    PATH_TO_MP4s
    
    with the path to wherever you stash your downloads, or a single dot (.) for current directory.

    So, for example, launch PowerShell, and:-
    Code:
    PS C:\Users\some.user>cd Z:\AnyStream\disney\Some.Title
    PS Z:\AnyStream\disney\Some.Title> ls
    
    
        Directory: Z:\AnyStream\disney\Some.Title
    
    
    Mode                 LastWriteTime         Length Name
    ----                 -------------         ------ ----
    ------        03/04/2021     00:45      524900316 Some.Title_S01E01_.mp4
    ------        03/04/2021     00:47      527526650 Some.Title_S01E02_.mp4
    ------        03/04/2021     01:37      527886615 Some.Title_S01E03_.mp4
    ------        03/04/2021     01:39      526319175 Some.Title_S01E04_.mp4
    ------        03/04/2021     01:41      529884906 Some.Title_S01E05_.mp4
    ------        03/04/2021     01:45      526732195 Some.Title_S01E06_.mp4
    ------        03/04/2021     01:47      526817790 Some.Title_S01E07_.mp4
    ------        03/04/2021     01:50      528575826 Some.Title_S01E08_.mp4
    ------        03/04/2021     01:52      528034184 Some.Title_S01E09_.mp4
    ------        03/04/2021     01:54      526905196 Some.Title_S01E10_.mp4
    ------        03/04/2021     01:56      526195807 Some.Title_S01E11_.mp4
    ------        03/04/2021     02:17      526961360 Some.Title_S01E12_.mp4
    ------        03/04/2021     02:20      527059484 Some.Title_S01E13_.mp4
    ------        03/04/2021     02:22      525375488 Some.Title_S01E14_.mp4
    ------        03/04/2021     02:24      530476690 Some.Title_S01E15_.mp4
    ------        03/04/2021     02:27      528540381 Some.Title_S01E16_.mp4
    ------        03/04/2021     02:29      527596756 Some.Title_S01E17_.mp4
    ------        03/04/2021     02:31      525865145 Some.Title_S01E18_.mp4
    ------        03/04/2021     02:34      527885878 Some.Title_S01E19_.mp4
    ------        03/04/2021     02:45      512530404 Some.Title_S01E20_.mp4
    ------        03/04/2021     02:47      521808110 Some.Title_S01E21_.mp4
    
    PS Z:\AnyStream\disney\Some.Title> $thresh=1000; gci . -Filter '*.mp4' | %{ $name=$_.FullName; C:\MediaInfo\MediaInfo.exe "--Inform=Video;%Duration% `nAudio;%Duration%" $name | %{ $l=$_.Split(' '); $d=$l[0] - $l[1];if([math]::abs($d) -gt $thresh){ $d/=1000; "!" + (&{if($d -lt 0){"V "}else{"A "}}) + $name + " |v|=" + ("{0:hh\:mm\:ss\,fff}" -f [timespan]::FromMilliseconds($l[0])) + " |a|=" + ("{0:hh\:mm\:ss\,fff}" -f [timespan]::FromMilliseconds($l[1])) + " ($d s)"};}}
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E01_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E02_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E03_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E04_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E05_.mp4 |v|=00:22:33,352 |a|=00:22:24,000 (9.352 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E06_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E07_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E08_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E09_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E10_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E11_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E12_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E13_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E14_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E15_.mp4 |v|=00:22:33,352 |a|=00:22:32,000 (1.352 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E16_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E17_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E18_.mp4 |v|=00:22:25,344 |a|=00:22:24,000 (1.344 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E19_.mp4 |v|=00:22:25,344 |a|=00:22:32,000 (-6.656 s)
    !A Z:\AnyStream\disney\Some.Title\Some.Title_S01E20_.mp4 |v|=00:21:45,304 |a|=00:21:44,000 (1.304 s)
    !V Z:\AnyStream\disney\Some.Title\Some.Title_S01E21_.mp4 |v|=00:22:09,328 |a|=00:22:16,000 (-6.672 s)
    

    You could wrap it up in a powershell script, but there are quirks to running powershell scripts, which most people (incorrectly) fix by creating a giant security hole (because that's the only way to "fix" the quirk that they could find on the "interwebs"), and this one liner is more than enough for this purpose.

    & voila! ;)
     
    Last edited: Apr 3, 2021
    whatever_gong82 likes this.
  5. donaf

    donaf Well-Known Member

    I'm seeing constant truncation issues of video and/or audio in Disney+ in AnyStream version 1.1.0.0 (as in case of Wild Russia, Season 1, Ep. 2; The Mandalorian Season 1, Ep. 1; and The Island At The Top Of The World). Please see attached log files.
     

    Attached Files:

  6. Jason Mulree

    Jason Mulree Member

    Based on my experience, the truncation issue is worse in this version than any other and appears to affect all providers.

    I previously downloaded some content using an earlier version of AS, though I cannot recall the version, it did not appear to suffer the truncation problem. I am using this as reference material.

    For instance, for a particular title, the downloaded title length in the previous version was 00:21:06 and appears complete and without truncation, but in the current version (1.1.0.0) the length of the same title is 00:20:56 and it is clearly cut off, so the video track, at least, is truncated. Interestingly, if I set the download speed to anything but unlimited, I consistently get a downloaded title with a length of 00:21:04, whereas if I set the download speed to unlimited, I consistently get a downloaded title length of 00:20:56. All of the download tests were conducted without restarting the application.

    I thought about providing logs, however I decided there are probably enough logs in this post already which cover various examples of this issue occurring. I mainly wanted to share that changing the download speed between unlimited and anything else appears to yield different, but consistent, results.
     
    Lowpro likes this.
  7. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    I've tested your theory, and it seems to make a difference on my end, but still not get the right result (title from D, actual duration: 23:13 according to the web player):-
    Code:
    $ for f in Tit*mp4; do ~/timeline "$f"; done|grep '^!'
    !V Title_S01E01_.x1.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    !A Title_S01E01_.x2.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:24.0000 (9.352 s)
    !A Title_S01E01_.x4.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:32.0000 (1.352 s)
    !V Title_S01E01_.xU.mp4 |v @ 640x480|=22:25.0344 |a AAC @ 128000 (en)|=22:32.0000 (-6.656 s)
    
    :)
     

    Attached Files:

  8. donaf

    donaf Well-Known Member

    Agreed... in downloading an episode from a TV series in Amazon Prime with the newest 1.1.0.0 version of Anystream, I just noted it had the truncation issue whereas the episode I downloaded before updating (meaning 1.0.9.0) did not have any truncation issue. Just as you indicated it appears to be a rather "global" issue and not just Disney+-centric.

    As a result, I just completely uninstalled 1.1.0.0 version of Anystream and reinstalled the prior release version of 1.0.9.0. I'll do without Disney+ until this is addressed. I'd rather have Amazon Prime working than nothing at all.

    I can appreciate significant changes were made to get the app to the 1.1.0.0 version, so I'll patiently wait for the bugs to get worked out a bit more before diving into the pool.
     
  9. Jason Mulree

    Jason Mulree Member

    Agreed that changing the speed still does not yield the correct result - from my testing, the best result obtained from changing the download speed to something other than unlimited was still a title that was 2 seconds shorter than the reference title, and suffers from obvious video track truncation.

    Assuming others observe similar behaviour, perhaps that changing the download speed yields different results might provide the developers with some insight into the issue ;)
     
    Last edited: Apr 4, 2021
  10. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    Indeed, my initial hunch was that it was a thread sync issue (based on non-deterministic behaviour) and your theory nudges me further along that line of thinking, but I've not seen the code. ;)
     
  11. Jason Mulree

    Jason Mulree Member

    The reason I tested different download speeds was because I was thinking it might have been, as you have suggested a thread sync issue, but possibly caused by some race condition that might be overcome by changing the download speed, especially since some users are reporting faster download speeds in the latest version where the truncation issue is seemingly more apparent.
     
  12. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    All we need now is a tarpit of a disk (I'm thinking good old floppies here) to write the output to, as mine gets written to a ramdisk over smb ;)
     
  13. donaf

    donaf Well-Known Member

    When this happened before in Amazon Prime to select titles, the workaround was to change the audio bitrate to a lower quality. That seemed to fix it. I don't know if that will work here though as this appears to be different in scope.
     
  14. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    I think it's a downloader problem: I've dug out "a tarpit of a disk" (RDX over USB2, the slowest I could find), and noticed that progress bar stops at 98% and AS goes into post-processing when I download the same title as in #7 at xU (the transition before was way too fast to spot that)...
     
  15. donaf

    donaf Well-Known Member

    That makes sense what you're saying. Sounds like AnyStream may be "jumping the gun" on post-processing and clipping audio/video before downloading is fully completed. Thanks for sharing your observations.
     
    whatever_gong82 likes this.
  16. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    Remains broken in 1.1.0.1, actual duration according to D's web player 23:32:-

    Code:
    !V Title_S01E01_.mp4 |v @ 640x480|=22:33.0352 |a AAC @ 128000 (en)|=22:40.0000 (-6.648 s)
    
     

    Attached Files:

  17. Ch3vr0n

    Ch3vr0n Translator NL & Mod

    i'll have to confirm this. D+, Series "Legends", EP 6, video gets cut off at 4:16 while audio continuous to play upto 4:25+
     
  18. Prospere

    Prospere RedFox Development Team

    This may be due to a slightly different track selection process. Different from that of the browser player used by D+.
    But the question is whether this affects the title playback. What I mean is whether the audio (in case audio is cut off first) is cut off while something is still happening in the video track or while watching the last seconds of credits.
     
    RedFox 1 likes this.
  19. malwa3e

    malwa3e New Member

    Video is cut off (mostly credits) and sound then continues to the end
     
  20. 0x0x0x0x0

    0x0x0x0x0 Well-Known Member

    It really does cut out:

    web player's last video frame
    wp.jpg

    downloaded last video frame
    dl.jpg

    I can upload full stts boxes for both tracks so you can compare them with what AS thinks they should be, if you're still logging samples :)