[Solved] Scheduled recordings completely disappearing

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

ZedThou
Junior
Posts: 25
Joined: Tue May 18, 2021 8:11 pm
United States of America

[Solved] Scheduled recordings completely disappearing

Post by ZedThou »

I'm using the launchpad Ubuntu 20.04 repository for nightly v32 builds, updating regularly. About a week or two ago, I began to periodically see that the list of scheduled recordings would go blank. Each time, I would restart the backend and they would come back. Looking through the backend log, I see errors that begin with

Code: Select all

May 19 08:59:00 mythtv mythbackend: mythbackend[176341]: I Scheduler scheduler.cpp:2309 (HandleReschedule) Reschedule requested for PLACE PrepareToRecord
May 19 08:59:00 mythtv mythbackend: mythbackend[176341]: E Scheduler mythdbcon.cpp:872 (prepare) Error preparing query: DROP TABLE IF EXISTS sched_temp_record;
May 19 08:59:00 mythtv mythbackend: mythbackend[176341]: E Scheduler mythdbcon.cpp:874 (prepare) Driver error was [2/4031]:#012QMYSQL3: Unable to prepare statement#012Database error was:#012The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.
So, I've increased the MySQL wait_timeout setting (default value was 8 hours), and hope that rectifies things. What I don't understand is why I'm seeing this behaviour now, and never did in the past.
Last edited by ZedThou on Thu May 20, 2021 1:38 pm, edited 1 time in total.
lostinignorance
Junior
Posts: 32
Joined: Fri Sep 01, 2017 2:58 am
United States of America

Re: scheduled recordings completely disappearing

Post by lostinignorance »

ZedThou wrote:
Wed May 19, 2021 8:31 pm
So, I've increased the MySQL wait_timeout setting (default value was 8 hours), and hope that rectifies things. What I don't understand is why I'm seeing this behaviour now, and never did in the past.
What file and location are you changing these settings in?
User avatar
bill6502
Developer
Posts: 2307
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: scheduled recordings completely disappearing

Post by bill6502 »

ZedThou
Junior
Posts: 25
Joined: Tue May 18, 2021 8:11 pm
United States of America

Re: scheduled recordings completely disappearing

Post by ZedThou »

lostinignorance wrote:
Thu May 20, 2021 1:29 am
What file and location are you changing these settings in?
I've put the setting

Code: Select all

wait_timeout = 2592000
in /etc/mysql/mysql.conf.d/mysqld.cnf, although it might be more "proper" to put personalized settings in /etc/mysql/my.cnf
bill6502 wrote:
Thu May 20, 2021 1:37 am
Note issue here too: viewtopic.php?f=36&t=4450&p=21883#p21880
I saw that thread, but didn't see any mention of the disconnect timeout in the error messages pasted there.
lostinignorance
Junior
Posts: 32
Joined: Fri Sep 01, 2017 2:58 am
United States of America

Re: scheduled recordings completely disappearing

Post by lostinignorance »

bill6502 wrote:
Thu May 20, 2021 1:37 am
Note issue here too: viewtopic.php?f=36&t=4450&p=21883#p21880
Thank you for that info, but I tried the command and I am getting an access denied. I am extremely new to mythtv from scratch as I really appreciated the pre-build in prior years as it made it dummy simple.
lostinignorance
Junior
Posts: 32
Joined: Fri Sep 01, 2017 2:58 am
United States of America

Re: scheduled recordings completely disappearing

Post by lostinignorance »

ZedThou wrote:
Thu May 20, 2021 1:57 am
I've put the setting

Code: Select all

wait_timeout = 2592000
in /etc/mysql/mysql.conf.d/mysqld.cnf, although it might be more "proper" to put personalized settings in /etc/mysql/my.cnf
I just modified that file. Have you been having good luck after making that modification? I am seeing the same thing that you are in my logs and all of a sudden, the upcoming schedule goes missing. I only notice it since I see my KODI front end showing schedule removals. This hasn't always been an issue for me either, its just something I've been battling with for a couple of weeks now and a reboot of the Ubuntu 20.04 instance fixes it till it disappears again.
ZedThou
Junior
Posts: 25
Joined: Tue May 18, 2021 8:11 pm
United States of America

Re: scheduled recordings completely disappearing

Post by ZedThou »

lostinignorance wrote:
Thu May 20, 2021 2:19 am
I just modified that file. Have you been having good luck after making that modification? I am seeing the same thing that you are in my logs and all of a sudden, the upcoming schedule goes missing. I only notice it since I see my KODI front end showing schedule removals. This hasn't always been an issue for me either, its just something I've been battling with for a couple of weeks now and a reboot of the Ubuntu 20.04 instance fixes it till it disappears again.
I've just made the setting change today, so will not know for a while whether it has an effect. At any rate, after modifying the cnf file, you'll want to restart mysql, the backend, and perhaps any frontends that are running.

Code: Select all

sudo systemctl restart mysql.service
sudo systemctl restart mythtv-backend.service
nouglywires
Junior
Posts: 20
Joined: Sun Nov 23, 2014 7:17 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by nouglywires »

Same thing happening here since May 13... It looks like mythfilldatabase runs OK - but after it completes, "upcoming recordings" is empty (in both local frontend and mythweb) (and nothing is recorded) unless I reboot. (Next time I'll try to stop/start mythbackend rather than rebooting).

I'm at [v31.0+fixes.202105172232.f8c59ee69d~ubuntu20.04.1]

UPDATE:
mythfilldatabase runs clean but leaves "upcoming recordings" empty.
Restarting the backend restores "upcoming recordings" - sudo systemctl restart mythtv-backend
Evidently the automatic restart isn't working.

Thanks for any suggestions
Last edited by nouglywires on Thu May 20, 2021 4:46 pm, edited 5 times in total.
crowston
Newcomer
Posts: 5
Joined: Mon Jun 15, 2020 7:12 pm
Canada

Re: Scheduled recordings completely disappearing

Post by crowston »

I'm having the same problem, apparently due to a system upgrade (Ubuntu 20.04): a new version of mysql on 13 May, from 8.0.23-0ubuntu0.20.04.1 to 8.0.25-0ubuntu0.20.04.1. But it's mysterious that if I restart the backend, it reschedules and the queries work fine but not after that. All of the tables pass the mysql check.

I have tried upping the wait_timeout and will see if that fixes it.

mythbackend --version
Please attach all output as a file in bug reports.
MythTV Version : v31.0-v31.0
MythTV Branch : fixes/31
Network Protocol : 91
Library API : 31.20200101-1
QT Version : 5.12.5
Options compiled in:
linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_firewire using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr using_ivtv using_joystick_menu using_libcec using_libcrypto using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_x11 using_libbluray_external using_xrandr using_profiletype using_systemd_notify using_systemd_journal using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_egl using_vaapi using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2 using_libmp3lame

mysql --version
mysql Ver 8.0.25-0ubuntu0.20.04.1 for Linux on x86_64 ((Ubuntu))
ZedThou
Junior
Posts: 25
Joined: Tue May 18, 2021 8:11 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by ZedThou »

crowston wrote:
Thu May 20, 2021 12:07 pm
I'm having the same problem, apparently due to a system upgrade (Ubuntu 20.04): a new version of mysql on 13 May, from 8.0.23-0ubuntu0.20.04.1 to 8.0.25-0ubuntu0.20.04.1.
Good catch. I had assumed the problem was somehow coming from mythtv v32, but you are seeing the same issue with v31. I also upgraded mysql packages from 8.0.23-0ubuntu0.20.04.1 to 8.0.25-0ubuntu0.20.04.1 on May 14.
spotlessyogurt
Newcomer
Posts: 9
Joined: Thu May 20, 2021 10:27 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by spotlessyogurt »

FYI I also noticed the exact same behavior; after upgrading to from 10.3.25-0ubuntu0.20.04.1 to 10.3.29-0ubuntu0.20.04.1 on May 12th, this started happening to me as well.

I've had to restart my mythtv-backend.service pretty much daily to get around this.

My theory is there is something that changed with session timeouts on the mariadb side.
BillK
Senior
Posts: 129
Joined: Tue Dec 09, 2014 9:49 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by BillK »

Can someone kindly explain how an 8-hour timeout might cause this problem?
spotlessyogurt
Newcomer
Posts: 9
Joined: Thu May 20, 2021 10:27 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by spotlessyogurt »

BillK wrote:
Fri May 21, 2021 2:53 pm
Can someone kindly explain how an 8-hour timeout might cause this problem?
Disclaimer - I have not inspected the Mythtv source code; this is my understanding from my limited exposure to SQL application coding from years ago. :)

Mythtv, a MySQL client, is connecting to the database server via a persistent connection; it logs in to the database and issues multiple queries for whatever it needs. It stays connected because it doesn't log-out. MySQL has an idle timeout that if you don't do anything for 8 hours, it decides your connection is wasting resources and terminates it. If the MySQL client (mythtv in this case) does not notice that its connetion has been terminated and tries to use the connection handle and other resources it was allocated on login, they will not work. Usually that shows up as errors on the mythtv side like "unable to execute query" and such. Basically, to mythtv, it looks like the database just "went away" mostly because I *assume* it's just not checking to see if the connection handle is still good (which is probably resource intensive and why it's not done that way).

Most applications take 1 of 2 approaches to this:
1 - they keep the connection alive, and after every query, they double-check to make sure it hasn't been disconnected due to some error. If it is, it tries to re-establish the connection, setup the query again, and execute it. The big problem with this method is that it's wasteful and begs the question; how many times do you retry before you assume the DB is *really* down?
2 - they establish a brand new connection handle with every query. The issue here is it's much more wasteful on both the client and the database server end to have to do this each time.

So I'm presuming mythtv is making the reasonable assumption that MySQL is tuned properly to not kick it out for whatever reason and is doing technique #1.

My guess is there is a bug in the most recent mysql/mariadb patch, possibly the client library. I've downgraded my client library for now and I'll find out in 2 days if my guess is right! :)
spotlessyogurt
Newcomer
Posts: 9
Joined: Thu May 20, 2021 10:27 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by spotlessyogurt »

BillK wrote:
Fri May 21, 2021 2:53 pm
Can someone kindly explain how an 8-hour timeout might cause this problem?
Oh, and I forgot to add - this happens to me sometime overnight. I think my last mythtv interaction with mysql tends to happen around 1 am and my next recording doesn't start until 2-3 pm the next day, so my evening recordings all get handled fine and by 2 pm my schedule looks empty. I'm presuming if I were to increase the timeout to 8 hours (or schedule something to record every 4 hours or so) this problem will go away.
nouglywires
Junior
Posts: 20
Joined: Sun Nov 23, 2014 7:17 pm
United States of America

Re: Scheduled recordings completely disappearing

Post by nouglywires »

The normal process is that mythfilldatabase clears out upcoming recordings and attempts to restart the backend, which builds and maintains the upcoming recordings list. Here's the mythfilldatabase log:

Code: Select all

===============================================================
| Attempting to contact the master backend for rescheduling.  |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted.                            |
===============================================================
2021-05-20 19:49:38.270340 N  mythfilldatabase run complete.
The problem here is that the backend restart step fails.

You can rebuild "upcoming recordings" by restarting the backend manually:
sudo systemctl restart mythtv-backend

This is not a database problem.

It's easy to verify this. Run mythfilldatabase and confirm that upcoming recordings is empty. Then restart the backend and see that the list is populated again.
Last edited by nouglywires on Fri May 21, 2021 5:14 pm, edited 1 time in total.
Post Reply