Scanning ts file for "actual" playback quality?

[wmorrison]

I am looking for a command, ffmpeg or other, that can scan a ts file for errors that will affect display, e.g. glitches that cause a loss of a few seconds (and possibly important dialog.) I don't care about recoverable errors (that is, where error correction succeeds in producing watchable output.)

I often have recordings that "glitch" despite not showing as damaged. I already have my quality set to 99 (can't remember the actual property name, but there's a post on here showing how to set this with an api call.)

I mainly want this for shows I will want to transcode with HandBrake, so I can scan without actually watching a whole 2-3 hour movie, and schedule a re-record if and when it is aired again.

I have not been able to find a command or combination of ffmpeg options that will give useful information. Even when I have two copies of a show, and have watched both, one is fine and the other has playback problems, I can't tell the difference with ffmpeg scanning for errors. The output is more or less the same. I probably don't know what to look for.

Any ideas?

[mythbuntubox]

In Europe we have a pretty old tool called ProjectX that may be useful, but not sure how well it will fare with US TV recordings. Might still be worth a look though:

https://sourceforge.net/projects/project-x/

[daraden]

After some digging, as far as I can tell the recording quality is only a measure of dts/pts gaps and continuity errors.

I wrote a scrip to count assorted errors from FFmpeg. To see if a recording may have any playback issues. Basically more errors while decoding would suggest there are going to be issues during playback.

Code: Select all

#!/usr/bin/env python3
import os
import sys
import subprocess
# FFmpeg error counter by Daraden
# Counts errors and warnings in video files
# Usage: /path/to/script /path/to/file/or/directory
# Logging usage: /path/to/script /path/to/file/or/directory /path/to/logfile

# Path to FFmpeg executable
ffmpeg = "ffmpeg"
# A/V file extensions to check
av_extensions = ('ts', 'mpg', 'mp4', 'mkv')
# Path to log file. can be changes to "/path/to/logfile" to log all usage
log_path = None
# for debug finding any missing errors and warnings
undefined_log = None


def string_in_list(string, _list):
    for value in _list:
        if value in string:
            return True
    return False


def write_file(path, string):
    with open(path, 'a') as dest:
        dest.write(string)


def output_data(path, string):
    if path:
        write_file(path, string)
    print(string)


def ffmpeg_error_scan(input_file):
    if sys.platform == 'win32':
        ffmpeg_command = [ffmpeg, "-v", "debug", '-loglevel', 'repeat+level+debug', '-err_detect', 'aggressive',
                          "-i", input_file, "-map", "0", "-f", "null", "NULL"
                          ]
    else:
        ffmpeg_command = [ffmpeg, "-v", "debug", '-loglevel', 'repeat+level+debug', '-err_detect', 'aggressive',
                          "-i", input_file, "-map", "0", "-f", "null", "-"
                          ]
    return subprocess.check_output(ffmpeg_command, stderr=subprocess.STDOUT, universal_newlines=True)


def run_count(input_file):
    print('working on file: {}'.format(input_file))
    run_command = ffmpeg_error_scan(input_file).splitlines()
    errors_dict = {'Error while decoding': 0, 'ac-tex damaged': 0, 'first mb_incr damaged': 0, 'mb incr damaged': 0,
                   'Invalid mb type': 0, 'incomplete frame': 0, 'end mismatch': 0, 'Warning MVs not available': 0,
                   'frame CRC mismatch': 0, 'skip with previntra': 0, 'invalid cbp': 0, '00 motion_type': 0,
                   'skipped MB': 0, 'slice mismatch': 0, 'overread': 0, 'slice below image': 0,
                   'Application provided invalid, non monotonically increasing dts to muxer': 0, 'qscale == 0': 0,
                   'Missing picture start code': 0, 'warning: first frame is no keyframe': 0, 'slice too small': 0,
                   'matrix damaged': 0, 'sequence header damaged': 0, 'ignoring seq ext': 0,
                   'ignoring GOP_START_CODE': 0, 'ignoring pic cod ext': 0
                   }
    warning_dict = {"corrupt decoded frame in stream": 0, 'PES packet size mismatch': 0}
    error_keys = errors_dict.keys()
    warning_keys = warning_dict.keys()
    error_list = [line for line in run_command if '[error]' in line]
    [[errors_dict.update({item: errors_dict[item] + 1}) for item in error_keys if item in line] for line in error_list]
    warning_list = [line for line in run_command if '[warning]' in line]
    [[warning_dict.update({item: warning_dict[item] + 1}) for item in warning_keys if item in line] for line in
     warning_list]
    # debug to find any errors/warnings not counted
    if undefined_log is not None:
        undefined_errors = [line for line in error_list if not string_in_list(line, error_keys)]
        undefined_warnings = [line for line in warning_list if not string_in_list(line, warning_keys)]
        if undefined_errors:
            write_file(undefined_log, '\n'.join(undefined_errors))
        if undefined_warnings:
            write_file(undefined_log, '\n'.join(undefined_warnings))
    input_sting = 'Results for file: {}\n'.format(input_file)
    warning_string = 'Warnings:\n{}'.format(''.join(['{} = {}\n'.format(k, str(v)) for k, v in warning_dict.items()
                                                     if v != 0
                                                     ]
                                                    )
                                            )
    error_string = 'Errors:\n{}'.format(''.join(['{} = {}\n'.format(k, str(v)) for k, v in errors_dict.items()
                                                 if v != 0
                                                 ]
                                                )
                                        )
    return ''.join((input_sting, warning_string, error_string))


if __name__ == '__main__':
    cmd_len = len(sys.argv)
    if cmd_len not in (2, 3):
        print('Missing or invalid input')
        sys.exit(1)
    else:
        file_path = sys.argv[1]
        is_path = os.path.exists(file_path)
        is_dir = os.path.isdir(file_path)
        is_file = os.path.isfile(file_path)
    if cmd_len == 3:
        log_path = sys.argv[2]
    if log_path is not None:
        valid_log_path = os.path.isdir(os.path.dirname(log_path))
        if not valid_log_path:
            print('Invalid logging path')
            sys.exit(1)
        else:
            print('logging to: {}'.format(log_path))
    if is_file and file_path.endswith(av_extensions):
        output_data(log_path, run_count(file_path))
    if is_dir:
        file_list = []
        for root, dirs, files in os.walk(file_path):
            file_list.extend([os.path.join(root, file) for file in files if file.endswith(av_extensions)])
        for item in file_list:
            output_data(log_path, run_count(item))
    if not any((is_dir, is_file)):
        print('Invalid file or directory')
        sys.exit(1)

[wmorrison]

Thanks. I'll check that out. I have a similar (not as detailed yet) Perl script. The hard part seems to be figuring out which part of the output is something that will actually affect playback quality and not just a warning that can be recovered. There's soooo much output.