50% of the data is unknown stream? Freesat

For discussion related to MythTV which doesn't belong in another forum.

Moderator: Forum Moderators

Post Reply
MozStoss
Newcomer
Posts: 10
Joined: Sat Jul 24, 2021 1:52 pm
British Indian Ocean Ter

50% of the data is unknown stream? Freesat

Post by MozStoss »

long term user and lover of mythtv.
This has been bugging me a couple of years.
my setup is UK Freesat

example
i record a program, then run a python script /ffmpeg that as well as renaming the file it strips out of the .ts subtitles, NAR etc just to leave the video and main audio. it does this in a couple of minutes via ffmpeg stream copy on a small raspberry pi

typically the reduced file size will be upto 50% smaller!!!!

ie 6GB->3GB sometimes and i cant see any noticeable reduction in video quality

when i use tools like mmpeg -i and mediainfo i can see that the extra NAR sound and subtitles and teletext dont equate to GB's of data just a couple of hundred MB.

so where did that extra data go?, the best i can guess when looking at the output of ffmpeg -i is that there are a bunch of streams that come up as unkown that are being stripped

heres some logs

Code: Select all

 mediainfo 7941_20210710183500.ts
General
ID                                       : 2050 (0x802)
Complete name                            : 7941_20210710183500.ts
Format                                   : MPEG-TS
File size                                : 5.75 GiB
Duration                                 : 1 h 54 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 7 162 kb/s

Video
ID                                       : 5400 (0x1518)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Format settings, GOP                     : M=8, N=24
Codec ID                                 : 27
Duration                                 : 1 h 54 min
Bit rate                                 : 6 162 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.119
Stream size                              : 4.95 GiB (86%)
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio #1
ID                                       : 5401 (0x1519)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : 129
Duration                                 : 1 h 54 min
Bit rate mode                            : Constant
Bit rate                                 : 384 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -1 s 56 ms
Stream size                              : 316 MiB (5%)
Language                                 : English
Service kind                             : Complete Main

Audio #2
ID                                       : 5402 (0x151A)
Menu ID                                  : 1 (0x1)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Codec ID                                 : 3
Duration                                 : 1 h 54 min
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Frame rate                               : 41.667 FPS (1152 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -960 ms
Stream size                              : 211 MiB (4%)
Language                                 : nar

Text
ID                                       : 5404 (0x151C)
Menu ID                                  : 1 (0x1)
Format                                   : DVB Subtitle
Codec ID                                 : 6
Duration                                 : 1 h 54 min
Delay relative to video                  : 4 s 528 ms
Language                                 : English

Other
ID                                       : 5403 (0x151B)-888
Menu ID                                  : 1 (0x1)
Format                                   : Teletext
Language                                 : English

Menu
ID                                       : 260 (0x104)
Menu ID                                  : 1 (0x1)
Duration                                 : 1 h 54 min
List                                     : 5400 (0x1518) (AVC) / 5402 (0x151A) (MPEG Audio, nar) / 5404 (0x151C) (DVB Subtitle, English) / 5403 (0x151B)-888 (Teletext, en) / 5401 (0x1519) (AC-3, English) / 2322 (0x912) () / 2321 (0x911) () / 2329 (0x919) () / 3851 (0xF0B) () / 3844 (0xF04) () / 3843 (0xF03) () / 3842 (0xF02) () / 3841 (0xF01) () / 3840 (0xF00) () / 3849 (0xF09) () / 3847 (0xF07) () / 3846 (0xF06) ()
Language                                 :  / nar / English / English / English

Code: Select all

mediainfo Darkest_Hour_\(2018\).ts
General
ID                                       : 1 (0x1)
Complete name                            : Darkest_Hour_(2018).ts
Format                                   : MPEG-TS
File size                                : 3.00 GiB
Duration                                 : 1 h 54 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 3 732 kb/s

Video
ID                                       : 256 (0x100)
Menu ID                                  : 1 (0x1)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Format settings, GOP                     : M=8, N=24
Codec ID                                 : 27
Duration                                 : 1 h 54 min
Bit rate                                 : 3 159 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.061
Stream size                              : 2.54 GiB (85%)
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 257 (0x101)
Menu ID                                  : 1 (0x1)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : 129
Duration                                 : 1 h 54 min
Bit rate mode                            : Constant
Bit rate                                 : 384 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Delay relative to video                  : -1 s 577 ms
Stream size                              : 316 MiB (10%)
Language                                 : English
Service kind                             : Complete Main

Menu
ID                                       : 4096 (0x1000)
Menu ID                                  : 1 (0x1)
Duration                                 : 1 h 54 min
List                                     : 256 (0x100) (AVC) / 257 (0x101) (AC-3, English)
Language                                 :  / English
Service name                             : Service01
Service provider                         : FFmpeg
Service type                             : digital television

Code: Select all

Input #0, mpegts, from '7941_20210710183500.ts':
  Duration: 01:54:59.90, start: 72802.961278, bitrate: 7160 kb/s
  Program 1
    Stream #0:0[0x1518]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x151a](NAR): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
    Stream #0:2[0x151c](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
    Stream #0:3[0x151b](eng,eng): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:4[0x1519](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:5[0x912]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:6[0x911]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:7[0x919]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:8[0xf0b]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:9[0xf04]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:10[0xf03]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:11[0xf02]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:12[0xf01]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:13[0xf00]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:14[0xf09]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:15[0xf07]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:16[0xf06]: Unknown: none ([11][0][0][0] / 0x000B)

Code: Select all

Input #0, mpegts, from 'Darkest_Hour_(2018).ts':
  Duration: 01:54:59.90, start: 1.400000, bitrate: 3730 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s

Code: Select all

-rw-r--r-- 1 nobody nogroup 5.8G Jul 10 21:30  7941_20210710183500.ts
-rw-r--r-- 1 moz    moz     3.0G Jul 24 13:18 'Darkest_Hour_(2018).ts'
Last edited by MozStoss on Sat Jul 24, 2021 3:42 pm, edited 1 time in total.
User avatar
kmdewaal
Developer
Posts: 641
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: 50% of the data is unknow stream? Freesat

Post by kmdewaal »

In your first log the video stream size is 4.95 GiB. In the second log the video stream size is 2.54 GiB. This then accounts for 2.41 GiB difference between the two files. The remainder of the 2.8 GiB difference in the file size will indeed be from the subtitles etc that your removed.
Question is then what is your exact ffmpeg command to achieve this!
MozStoss
Newcomer
Posts: 10
Joined: Sat Jul 24, 2021 1:52 pm
British Indian Ocean Ter

Re: 50% of the data is unknown stream? Freesat

Post by MozStoss »

thanks for the reply
im happy im getting exactly the file i want, im just bemused as to why its so smaller, its not just this but all that i record

ive just manually run to confirm ffmpeg -i 7941_20210710183500.ts -c copy -map 0:v:0 -map 0:4 newfile.ts

Code: Select all

moz@raspberrypi:~/USB/MYTH-TV $ ls -lh 7941_20210710183500.ts newfile.ts
-rw-r--r-- 1 nobody nogroup 5.8G Jul 10 21:30 7941_20210710183500.ts
-rw-r--r-- 1 moz    moz     3.0G Jul 24 16:52 newfile.ts

Code: Select all

moz@raspberrypi:~/USB/MYTH-TV $ ffmpeg -i 7941_20210710183500.ts -c copy -map 0:v:0 -map 0:4 newfile.ts
ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Raspbian 8.3.0-6+rpi1)
  configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
  WARNING: library configuration mismatch
  avutil      configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  avcodec     configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  avformat    configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  avdevice    configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  avfilter    configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  avresample  configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  swscale     configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  swresample  configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  postproc    configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] decode_slice_header error
[h264 @ 0x113b860] no frame!
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] decode_slice_header error
[h264 @ 0x113b860] no frame!
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] decode_slice_header error
[h264 @ 0x113b860] no frame!
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] SPS unavailable in decode_picture_timing
[h264 @ 0x113b860] non-existing PPS 0 referenced
[h264 @ 0x113b860] decode_slice_header error
[h264 @ 0x113b860] no frame!
[h264 @ 0x113b860] mmco: unref short failure
    Last message repeated 1 times
[mpegts @ 0x11371d0] Could not find codec parameters for stream 5 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 6 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 7 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 8 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 9 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 10 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 11 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 12 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 13 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 14 (Unknown: none ([11][0][0][0] / 0x000B)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 15 (Unknown: none ([11][0][0][0] / 0x000B)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mpegts @ 0x11371d0] Could not find codec parameters for stream 16 (Unknown: none ([11][0][0][0] / 0x000B)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from '7941_20210710183500.ts':
  Duration: 01:54:59.90, start: 72802.961278, bitrate: 7160 kb/s
  Program 1
    Stream #0:0[0x1518]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x151a](NAR): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s
    Stream #0:2[0x151c](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
    Stream #0:3[0x151b](eng,eng): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:4[0x1519](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:5[0x912]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:6[0x911]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:7[0x919]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:8[0xf0b]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:9[0xf04]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:10[0xf03]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:11[0xf02]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:12[0xf01]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:13[0xf00]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:14[0xf09]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:15[0xf07]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:16[0xf06]: Unknown: none ([11][0][0][0] / 0x000B)
Output #0, mpegts, to 'newfile.ts':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Stream #0:1(eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:4 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=172458 fps=1644 q=-1.0 Lsize= 3142397kB time=01:54:59.45 bitrate=3731.1kbits/s speed=65.8x
video:2546854kB audio:323368kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.482687%
User avatar
kmdewaal
Developer
Posts: 641
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: 50% of the data is unknown stream? Freesat

Post by kmdewaal »

I have done a quick test on a few Freesat/Astra2 recordings.
Result for channel 175, Smithsonian HD, a recording of Aerial America, H264 video: file size remains about the same.

Code: Select all

[klaas@zolder show_names]$ ls -l aa*
-rw-rw-r--. 1 klaas klaas 3927519092 Jul 24 20:19 aanew.ts
-rw-r--r--. 1 klaas klaas 3907259084 Jul 24 20:17 aa.ts
Result for channel 103, ITV HD, a recording of Tipping Point, H264 video: file size remains about the same.

Code: Select all

[klaas@zolder show_names]$ ls -l itv*
-rw-rw-r--. 1 klaas klaas 3109868740 Jul 24 20:33 itvnew.ts
-rw-r--r--. 1 klaas klaas 3356808244 Jul 24 20:32 itv.ts
But, on channel 101, BBC One Londen, a recording of BBC News, MPEG2 video: file size about halved!

Code: Select all

klaas@zolder show_names]$ ls -lh b*
-rw-rw-r--. 1 klaas klaas 885M Jul 24 20:12 bbc1new.ts
-rw-r--r--. 1 klaas klaas 1.7G Jul 24 20:06 bbc1-sd.ts
As in the original post, the file size reduction is almost completely due to a reduction in the size of the video stream.

So the issue is reproducible but not for all Freesat channels.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: 50% of the data is unknown stream? Freesat

Post by paulh »

Most likely it is the MHEG streams for interactive TV. You will most likely see this on BBC channels since they have the RED button service.

If you play a recording that still has those streams and you press the RED (F1) key you should see the interactive stuff start up even though it's a recording 8-)

I thought there was a setting you can use to tell the recorder to not save those streams but for the life of me can't find it or remember where it is :oops: .
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: 50% of the data is unknown stream? Freesat

Post by paulh »

OK found it :)

It's under Recording Profiles -> Hardware DVB Encoders -> Default -> Recording Type and change it to TV Only.
MozStoss
Newcomer
Posts: 10
Joined: Sat Jul 24, 2021 1:52 pm
British Indian Ocean Ter

Re: 50% of the data is unknown stream? Freesat

Post by MozStoss »

thanks for testing kmdewaal, I think you are onto something with it being just BBC, i'm away from my setup a few days, Grrr so cant test until i get back

paulh, that sounds like it could be ticket and works with kmdewaal's findings about BBC :)

im looking forward to getting home and playing and freeing up future storage space

thanks fellow, mythtv'rs :)
User avatar
kmdewaal
Developer
Posts: 641
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: 50% of the data is unknown stream? Freesat

Post by kmdewaal »

I did a quick test with the "TV Only" setting as given by paulh and can confirm that this does indeed work as expected; only the video and audio streams are then recorded. The group of "Hardware DVB Encoders" consists of all devices with /dev/dvb/ drivers. This is a useful feature to add to the Sat>IP support because for that driver this choice is not available.

A bit of searching showed this comment https://code.mythtv.org/trac/ticket/1341#comment:17, already 15 years ago, describing this issue.
Post Reply