USB DVB tuners and the Raspberry Pi - a warning

For discussion of topics specific to MythTV on Raspberry Pi devices
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

EDIT: I STARTED THIS THREAD AFTER SOME EXPERIMENTS LED ME TO BELIEVE THERE WAS AN INTERMITTENT PROBLEM WITH USING USB DVB/T2 TUNERS ON THE RPi. FURTHER EXPERIMENTS HAVE REVEALED THAT THE RPi IS NOT THE PROBLEM. THE LATEST INFO IS AT THE END OF THE THREAD.

This is a heads-up for anyone experiencing what seems to be impulsive signal interference using USB DVB/T2 tuners on a RPi4 backend - it may very well not be interference at all. For the gory details, read on...

I have been spending a great deal of time and effort trying to track down the source of occasional brief moments of what looks exactly like impulsive interference on my recordings. These are recordings from UK Freeview HD channels. They are generally fine, but every now and then there would be a brief moment of pixellation.

Naturally, I started by assuming there was some interference on the signal being delivered by the aerial, even though I have good signal strength, living just 4 miles from a main transmitter. I tried a log periodic instead of Yagi aerial, as they are reported to be more resistant to impulsive interference. I tried adding an AT800 filter. I tried putting in some attenuation in case the tuner was overloaded. Nothing helped.

Next I suspected the tuner itself - a cheap Astrometa DVB/T2 USB device. I also had a Hauppauge WinTV-HD USB tuner lying about. Rather than just swapping it in, I built a second Myth backend on an old laptop and set it up to record exactly the same programmes as the main RPi4B system, from the same aerial via a passive splitter on the aerial feed. This meant I could compare the quality of recordings from the two tuners.

Success! - Apparently. The recordings on the laptop using the Hauppauge tuner were free of interference. So I swapped the Hauppauge and Astrometa tuners over (and their cabling, just in case the cabling itself was involved) and set up the comparison again. But it was STILL the laptop recordings (now using the Astrometa tuner) that were fine, while the RPi4 recordings (now using the Hauppauge tuner) had interference.

Next I wondered if perhaps the RPi4B was not providing sufficient power for the SSD and the tuner (which were both connected directly to the RPI4's USB ports), so I added a powered USB hub to supply the tuner and the SSD. Still no luck.

I have come to the conclusion that it must be something to do with the OS or LinuxTV system or device drivers on the RPi4 that's giving the problem. The laptop (running Linux Mint) has a Celeron CPU, whereas the RPi4 of course has an ARM CPU. Presumably the device drivers for the tuners will be different. (I know the firmware is the same).

The RPi4 is running Buster rather than Bullseye, which I suppose might be a factor. But I've got to the point now where I'm fed up installing OS's and building Myth backends for testing purposes, so I'm just going to abandon the RPi4 and use an x86 machine. It seems to be the path of least pain.
Last edited by cliveb on Tue Mar 28, 2023 9:22 am, edited 1 time in total.
User avatar
pgbennett
Developer
Posts: 503
Joined: Mon Apr 27, 2015 5:41 pm
United States of America

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by pgbennett »

Have you validated that it is the recording that is faulty and not the playback?

Likely it is caused by dropped data. If data comes in too fast for it to be processed and some bytes are sometimes dropped, you will see occasional minor pixelation.
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

pgbennett wrote:
Wed Mar 08, 2023 6:08 pm
Have you validated that it is the recording that is faulty and not the playback?
Yes, absolutely.
I downloaded the .TS files from the two systems to my main laptop and played them in VLC.
Have repeated this comparison on multiple recordings.
The recordings made on the RPi4 had the pixelation, while the ones from the test laptop did not.

NB. Just in case you're wondering, the test laptop is not a high powered device. Its CPU is a Celeron U4100 running at 1.3GHz, so probably less powerful than the RPi4.
jksj
Senior
Posts: 148
Joined: Thu Feb 13, 2014 7:53 pm
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by jksj »

Presumably you were running a full backend on the Pi.
I get no issues using a Pi 4 to handle 2 DVB tuners when just running SAT/IP (https://github.com/catalinii/minisatip)on the machine sending the output to a NUC based backend by ethernet. https://www.mythtv.org/wiki/Channel_Sca ... EIP_tuners
Having said that I had to change the operating mode of the USB tuners from Isochronous transport mode to bulk transport mode. https://github.com/b-rad-NDi/eeprom-tinker and make sure that there were enough free UDP packet buffers in the NUC to handle the network traffic from the Pi.
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

jksj wrote:
Thu Mar 09, 2023 9:55 am
Presumably you were running a full backend on the Pi.
Yes, a full backend.
Maybe I've been unlucky, but all the evidence is zeroing in on the Pi.
Maybe it's just the Astrometa and Hauppauge tuners that have problems.
(I will also point out that SD recordings from these tuners are fine; it's only HD recordings that have problems).
I did also have the tuners in bulk transfer mode - that's something I discovered some time back.
I guess it's possible that perhaps the Pi itself has some kind of fault, but it appears to operate correctly in all other respects.

I'm sure others might have been able to get things working faultlessly on a Pi, but I've lost the will to experiment any more, and will just switch the backend to an x86 machine. My purpose in posting this was just to let people know that what looks like signal interference might not actually be that.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by paulh »

My guess would be since you only see it with HD and not SD recordings is the bandwidth requirements to both read the data from the USB device and write the data to the drive are just too much for the of the single USB hub the Pi's have.

EDIT
It could also be your USB device just sends all the data from a multiplex over USB and lets the software driver filter the data to get just the PIDs that are required (I believe most most cheap cards and devices do that since the hardware is not capable of doing the filtering). Since the HD multiplexes send more data that the SD ones then that would make the problem more visible on those multiplexes
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

paulh wrote:
Thu Mar 09, 2023 5:29 pm
My guess would be since you only see it with HD and not SD recordings is the bandwidth requirements to both read the data from the USB device and write the data to the drive are just too much for the of the single USB hub the Pi's have.
I guess that's a possibility, since the test backend on the laptop has its disk attached by SATA rather than USB.

BUT... there have been times when multiple programmes have been recorded simultaneously and without any pixelation, while I have seen cases of pixelation where just one programme is being recorded.
paulh wrote:
Thu Mar 09, 2023 5:29 pm
It could also be your USB device just sends all the data from a multiplex over USB and lets the software driver filter the data to get just the PIDs that are required (I believe most most cheap cards and devices do that since the hardware is not capable of doing the filtering). Since the HD multiplexes send more data that the SD ones then that would make the problem more visible on those multiplexes
Well yes, that's the whole point of multirec, isn't it?
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by paulh »

It's a good point I'm not sure how Myth uses the DVB API to get all the PID's it needs for multirec.

From my limited knowledge I think it goes something like this. Myth uses to DVB API to get all the associated PID's (video stream, audio stream(s), subtitle streams etc) for a particular channel using it's Service ID. The API then asks the driver for the device for the data streams (PID's) it want. It's up to the driver whether it reads the entire data for the multiplex and does the PID filtering in software or it tells the hardware to do the filtering and only gets the data streams it's wants back from the hardware.

I would think for multirec you just ask the API for another SID and if it's available you also get the data for all the streams associated with the second channel you want which would only work if the channel was on the same multiplex as the first one. Myth knows which channels (SID's) are on which multiplex so knows which channels can be recorded together.
User avatar
dnalorernst
Developer
Posts: 103
Joined: Mon Feb 17, 2020 8:03 pm
Austria

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by dnalorernst »

I remember years ago I had to put the CPU Frequency Governor to use always a fixed (high) frequency. IIRC, it was set to 'Performance'.
Otherwise, I got continuity errors on DVBS2 (h.264) during recording.
It was not the CPU load causing these errors, it was the frequency switching.
See if something like

Code: Select all

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
is set.
I put a small script triggered by the 'Recording Pending' event to manage this.
Be aware on correct cooling the Raspi 4.
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

dnalorernst wrote:
Sun Mar 12, 2023 3:15 pm
I remember years ago I had to put the CPU Frequency Governor to use always a fixed (high) frequency. IIRC, it was set to 'Performance'.
Otherwise, I got continuity errors on DVBS2 (h.264) during recording.
It was not the CPU load causing these errors, it was the frequency switching.
See if something like

Code: Select all

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
is set.
I put a small script triggered by the 'Recording Pending' event to manage this.
Be aware on correct cooling the Raspi 4.
That is interesting. Presumably you're saying that if the governor chooses to change the cpu frequency at an inconvenient moment, there is a brief interruption to extracting the relevant data from what's coming in from the tuner? It's also interesting that your experience is with DVB/S2 and mine is DVB/T2, both of which are H.264 format.

I checked my scaling governor and it's set to "ondemand". I'll try setting it to "performance" to see if that makes a difference and keep an eye on cpu temperature. (My RPi4 is passively cooled with the standard official "large" heatsink).
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

cliveb wrote:
Mon Mar 13, 2023 10:23 am
I checked my scaling governor and it's set to "ondemand". I'll try setting it to "performance" to see if that makes a difference and keep an eye on cpu temperature. (My RPi4 is passively cooled with the standard official "large" heatsink).
Update: Having set the governor to "performance", tonight there was another glitch on a recording (test system was fine).

So whatever is behind the RPi's occasional glitches remains a mystery.
User avatar
dnalorernst
Developer
Posts: 103
Joined: Mon Feb 17, 2020 8:03 pm
Austria

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by dnalorernst »

I get at most one continuity error per recording, and this is immediately after the recording starts. But i do not bother, because it is in the pre-roll phase of the recording.
You have not stated if the performance governor setting did some improvements.
I think it is now time to post some logs.
The code from dtvrecorder.cpp indicates the correct log setting (mythbackend.log):

Code: Select all

LOG(VB_RECORD, LOG_WARNING, LOC +
            QString("PID 0x%1 discontinuity detected ((%2+1)%16!=%3) %4%")
                .arg(pid,0,16).arg(old_cnt,2)
                .arg(tspacket.ContinuityCounter(),2)
                .arg(erate));

Code: Select all

mythbackend --setverbose record --setloglevel warning
should provide a very condensed log, otherwise use

Code: Select all

mythbackend --setverbose record --setloglevel info
A thump question, I know: Are the 2 DVBT tuners connected to USB 2 ports ?
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

dnalorernst wrote:
Wed Mar 15, 2023 6:20 pm
I get at most one continuity error per recording, and this is immediately after the recording starts. But i do not bother, because it is in the pre-roll phase of the recording.
You have not stated if the performance governor setting did some improvements.
If when you say "some improvements" you mean did it reduce the problem at all, it's impossible to say because the scale of the probem is so variable. One day it'll be fine, another day quite bad. All I can say is that changing the governor to "performance" did not *eliminate* the problem, which is the goal. And I certainly sometimes get multiple issues in a single recording. Whether these issues are "continuity errors" I can't say - that have the appearance of pixelation due to interference of some sort.
dnalorernst wrote:
Wed Mar 15, 2023 6:20 pm

I think it is now time to post some logs.
Well, I've now rebuilt my backend on an x86 machine (a Lenovo M73), but what I will do is put together another test backend on the RPi4 and set up the logging. Watch this space.
dnalorernst wrote:
Wed Mar 15, 2023 6:20 pm
A thump question, I know: Are the 2 DVBT tuners connected to USB 2 ports ?
Yes, the DVB tuners are on USB2 ports. (Well, at some point they were on the RPi's USB3 socket, but as USB2 devices they just get attached to the USB2 hub. They were also at some point attached via a powered USB3 hub - which I did in case there was a power supply issue. But that made no difference).
cliveb
Senior
Posts: 131
Joined: Fri Jan 08, 2016 9:59 am
Great Britain

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by cliveb »

cliveb wrote:
Sat Mar 18, 2023 1:20 pm
Well, I've now rebuilt my backend on an x86 machine (a Lenovo M73), but what I will do is put together another test backend on the RPi4 and set up the logging. Watch this space.
So: having run the test backend on the RPi, there was one instance of "discontinuity detected" in the log.
However, the recording at that point looks fine. (Although the exact timestamp of the discontinuity is at a significant scene change, so I suppose if there was any glitch it might not have shown up. That said, there was no pixelation, which is what every glitch up to now has exhibited).

In the meantime, I've seen a couple of pixelations on the new x86 backend, and no corresponding interference on the test RPi backend.
The only piece of hardware that is common to all the system builds that have pixelation is the disk itself (a Western Digital Blue SSD). Up until now the WD SSD was on the production RPi, and the test systems were using another disk. When I rebuilt the production system on the x86 machine, I used the WD SSD. So I'm beginning to think that perhaps there's a fault in the SSD. Maybe there are some bad blocks which haven't been mapped out? I'll look for a Linux SSD health check application to try on it.

If it does turn out to be the SSD, then my suspicions about the RPi were wrong.
User avatar
kmdewaal
Developer
Posts: 640
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: USB DVB tuners and the Raspberry Pi - a warning

Post by kmdewaal »

You can use gnome-disks to read the SMART data of the disk. This might give an indication if and why the disk is failing.
Post Reply