AnyBDMV: Movie recovery product

Discussion in 'General Chat' started by markfilipak, Sep 17, 2018.

  1. markfilipak

    markfilipak Well-Known Member

    Back in the floppy disk days, I had a tool for recovering bad floppies. It was pretty amazing. It scoreboarded read-data & errors, and did it persistently, "washing machine" style, until it had accumulated sufficient read-data to statistically rebuild a "best guess" of what the original data was. It actually worked quite well.

    The Need:
    Worldwide, DVD & BD movie discs are going bad. Often, the discs are old and out of print, so can't be replaced. I think people would buy AnyBDMV. It is needed.

    AnyBDMV == AnyDVD HD with persistent, statistics-based recovery.

    My Experience:
    The ending 20 minutes of my "BLADE RUNNER FINAL CUT" went bad -- it would break up and freeze in my home theater and in my computer. I recovered the entire movie (*) by, 1: running AnyDVD over & over (manual "washing machine"), then 2: using its reports (**) and a powerful binary comparision tool. I scoreboarded hand-crafted statistics and rebuilt the movie.

    My 1st 'run' reported 13 separate 'sector-groups' with errors (11 of which were 30 sectors in extent and 2 of which were 60 sectors in extent) in the final 20% of the movie. In the end, I had reduced that to 6 separate 'sector-groups' in error (only 1 of which was 60 sectors in extent) very near the end of the movie.

    (*) A few scan lines of a few frames of the ending credits could not be recovered, but AnyDVD apparently padded out the missing data so that the frames didn't break up.

    (**) The reports are in 'sector-groups' of 30 sectors each. That resolution is way too course: The scoreboard needs finer granularity. Note: There are 2048 bytes per sector, 30 sectors per 'sector-group'.

    Actually, if short, visual/audible data corruption is pretty unnoticable provided frames don't break up.

    Warm Regards,
    Mark.
     
  2. Ch3vr0n

    Ch3vr0n Translator NL & Mod

    Anydvd already let's you retry sectors with read errors and skip them too. Chances of your imagined product becoming reality, I'd say none whatsoever.

    Verstuurd vanaf mijn Nexus 6P met Tapatalk
     
  3. markfilipak

    markfilipak Well-Known Member

    Automatic operation, with 'sector-groups' that are much, much smaller, would make it worth the cost (whatever that would be). As it is, I used AnyDVD to do what I wanted, but the time required and the GB-comparisons and editing was not fun. Just think about it. It's your call, not mine.

    Regards,
    Mark.
     
    Last edited: Sep 17, 2018
  4. Ch3vr0n

    Ch3vr0n Translator NL & Mod

    It ain't mine either. I'm just a volunteer mod, I don't work for redfox or help develop any product.

    Verstuurd vanaf mijn Nexus 6P met Tapatalk
     
  5. markfilipak

    markfilipak Well-Known Member

    Of course you don't. Nobody does. RedFox doesn't exist.
     
  6. Jamie

    Jamie Well-Known Member

    Files on hard disks can go bad too. So there is no guarantee that the ripped movies will not become defective over time.
     
  7. markfilipak

    markfilipak Well-Known Member

    I don't see the relevance, Jamie. Are you proposing that RedFox go into the insurance business?
     
  8. Jamie

    Jamie Well-Known Member

    I shouldn't have quoted that post an over sight.

    I'm just saying that if your dvd/bluray disk goes bad that there is no guarantee that the movie will stay good on the hard/disk. If you can recover the bluray/dvd disk you're good. if not you're going to have to purchase a new disk if it's available but you will probably have to pay big bucks if the movie is out of print. I keep a backup of all my hard drives containing my movies offsite. Hopefully both copies don't break at the same time.
     
  9. mmdavis

    mmdavis Well-Known Member

    Sounds like a business opportunity for you.
     
    FurryGuy likes this.
  10. Pete

    Pete Forum Admin Staff Member

    I'm sorry, but reality stands in the way of applying a finer granularity.
    On Blu-ray discs, 32 sectors (not 30) are grouped together to an ECC unit (Error Correction Code). You can either get all data from all of those 32 sectors (64kB) or not a single bit at all, because that's how the hardware works.
    It can detect whether those 64kB are intact or not (per 64kB there are 848 bytes of additional ECC-information, the method is not the same, but similar in principle to Reed Solomon codes used on DVDs and QR scan codes, for whoever cares to learn how error correction works).

    If there's something broken, it will be possible to reconstruct the whole 64kB, as long as not more than "x" bits are broken. The number varies and depends on the layout of the errors and whether it's a burst or a spurious error.
    But an ECC block can actually take quite some damage and still be correctable.
    If too many bits are broken to reconstruct the 64kB, then it will be unknown, which bits are broken, the whole 64kB are useless and a read-error gets reported.

    Absolutely nothing can be done about it - it will always be groups of 32 sectors in a row to either fail or succeed (and your "60 sectors" are simply two ECC blocks in a row failing and should really be 64 sectors).

    BTW: it is totally common, that the Blu-ray Disc drive reconstructs sectors (rather: ECC groups), it happens all the time. There is a continuous stream of misread bits, the whole magic is simply to ensure, that the percentage of bad bits remains below a threshold. ECC then still guarantees 100% correct bits.
    Sometimes it rises above and then it's an uncorrectable read error.

    Especially with video data a single flipped bit is typically enough to destroy the complete frame, so there's no point in "salvaging as much as possible". If you know, that at least a single bit is broken, you have to properly discard the data and inform the player instead of sneaking in partially garbled data.
    The player can simply skip to the next frame, which is much less disruptive than displaying a broken frame (succeeding frames would usually be broken as well, because of frame references).

    So the way I see it, the only thing AnyDVD can do, is retry a couple of times and then abandon the area. Which it already does. It places NULL sectors wherever it couldn't get data - these will automatically be skipped, because there is no PID to demultiplex them.
     
    nebostrangla likes this.
  11. markfilipak

    markfilipak Well-Known Member

    Yes, 32 sectors makes more sense. I must have simply been misled by the log (for example: "Read Error Sectors: 11293050-11293079") into thinking it was 30 sectors. My bad.
    Suggestion: Hardware usually supports raw reads. If so, then for those sectors that fail ECC, do persistent raw reads, scoreboard the raw data, make statistical best guess.
    Obviously, rereading makes a difference. It did for me. I persistently reran the ripper and it made a huge difference.

    This is a much needed product. RedFox already has most of the pieces with which to build it.

    People (and stores) will buy this product just as they buy disc refinishers.

    PS: I suspect people will buy this BEFORE trying a disc refinisher for 2 reasons:
    1, It wouldn't "wear" the disc ("failures to correct" are nondestructive), and
    2, It can get better over time through software updates.
     
    Last edited: Sep 18, 2018
  12. markfilipak

    markfilipak Well-Known Member

    I'd like to provide just a few more arguments. As with all forum posts, kindly disregard if you find me annoying...

    AnyBDMV (or whatever you would name it) would provide a product that you could sell in the legitimate market. ..."And, oh, by the way, we have this other product (that we can't name) that will back up your existing Blu-rays before they go bad." Thus, it could provide an entry into the legitimate market. Right now, people learn about your products essentially by word-of-mouth, only.

    AnyBDMV would bolster your (totally proper) contention that your products are meant to protect movie buyers' investments, not to promote piracy.
     
    Last edited: Sep 18, 2018
  13. StoneyJSG

    StoneyJSG Well-Known Member

    Why not make and market AnyBDMV yourself?
     
    FurryGuy likes this.
  14. markfilipak

    markfilipak Well-Known Member

    I'll be 72 years-old next week. I'm retired. I have fun in retirement. I'm happy being retired, and I don't want to spend my time pulling my hair out.

    I have no BD-play or -decryption software library from which to draw, so I'd have to develop all that.

    In short, I'd probably be dead before I got working code.
     
  15. James

    James Redfox Development Team Staff Member

    AnyDVD ripper reads chunks of 30 sectors, but this is unrelated to ECC grouping. This is done to increase the performance of the real-time decryption & remastering.

    I don't know any drive supporting raw reading on Blu-ray discs.
     
  16. markfilipak

    markfilipak Well-Known Member

    Okay. It was an idea. If you say it can't be done, then it can't be done.

    Warm Regards, James,
    Mark.
     
  17. markfilipak

    markfilipak Well-Known Member

    Hey James,
    I had never tried it before... I just copied a BD movie's '/BDMV/' to my hard disk. If that wasn't a raw read, what was it?

    Thanks!
     
  18. James

    James Redfox Development Team Staff Member

    A read.
     
  19. FurryGuy

    FurryGuy Well-Known Member

    @markfilipak,

    You keep babbling on as if you know more about decrypting and reading DVD and Bluray discs than the developers of AnyDVD.

    Give it a rest, if they don't find your nonsense credible, then no-one will.
     
    amazing likes this.
  20. Pete

    Pete Forum Admin Staff Member

    Please try being a bit less condescending.
    His arguments do make sense, even if they don't work that way due to technical reasons.
    If we're going to shut everyone up, because their output is not immediately nobel-prize-worthy, nobody will ever mouth a single idea ever again and we're back to chiselling rocks.