Suggestions and Tips for upgrading from v30 -> v32?

For discussion related to MythTV which doesn't belong in another forum.

Moderator: Forum Moderators

Post Reply
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

Hey everyone,

Soimeone recently told me that my v30 install will likely stop working sometime soon, as the old method of grabbing schedule updates is about to be discontinued. I don't know the details, but I am thinking maybe it is time to upgrade.

Right now, I have a dedicated v30 backend running inside of an Ubuntu 18.04 LTS Server Edition LXC container install on my Debian based KVM/LXC combo server. The server is headless, but I have installed an LXDE desktop in the container, and manage it remotely using X2Go. For a dedicated backend, this solution has worked well for many years now, ever since I switched my server from ESXi to KVM+LXC in 2014.

I have also heard that in going from v30 to newer versions, it isn't quite as straight forward as in the past, as some things have changed, that may require reconfiguration.

I wonder if anyone has any suggestions on how to accomplish this.

Is it even worth trying to upgrade, or should I just export my database and existing recordings, create a new container, maybe on a newer version of Ubuntu, like 20.04 LTS or maybe even 22.04 LTS. I'm usually a "use the oldest LTS version still supported" kind of guy, but maybe for the MythTV install, it is worth going a little newer?

Can anyone link me to information on the following two topics?

1.) What I need to reconfigure when upgrading to a newer version than v30; and

2.) The details around the changes on Schedules Direct I need to be aware of?

I'd appreciate any input on this topic, as well as any suggestions anyone might have.

Thank you,
Matt
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
User avatar
heyted
Senior
Posts: 301
Joined: Sun Jun 08, 2014 2:14 am
Location: South Florida
Contact:
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by heyted »

I usually do a fresh install of Ubuntu and use the Mythbuntu MythTV Updates PPA to install MythTV using MCP. Ubuntu 22.04.1 LTS will be released on approximately August 11. This video may be helpful if using backend-setup to configure the grabber for Schedules Direct. The video uses the tv_grab_zz_sdjson grabber which is compatible with MythTV v32. Alternatively, the procedure for using the command line to configure the grabber is described on the wiki.
Ted | My blog
mtvf2894
Junior
Posts: 81
Joined: Sat Apr 10, 2021 2:33 pm
Great Britain

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mtvf2894 »

Suggest reading the Major Changes for v31 and v32 (perhaps skim the other changes):

https://www.mythtv.org/wiki/Release_Notes_-_31
https://www.mythtv.org/wiki/Release_Notes_-_32

If you use hardware accelerated playback there may be issues. Weirdly for me v31 / 20.10 worked sort of okay but since going to v32 / 22.04 hardware accelerated playback has been very flaky. Also worth noting that Bookmarks have also changed https://github.com/MythTV/mythtv/commit/c6569589a8

I usually do a clean install of both the OS and MythTV but last time around I did a clean install of the OS and restored the MythTV database using a backup. This saved me from having to reconfigure MythTV which saved many hours. If you go this route make sure you check mythbackend.log to ensure the database schema updates have been successfully executed when you first run the backend.

I would use 22.04 why make future work for yourself (assuming it works with your containerization).
Current: v32 now v33.1 / Lubuntu 22.04 / AMD APU
Previous: v31 / Lubuntu 20.10 / AMD APU
Historic: Mythbuntu / NVIDIA GFX
User avatar
bill6502
Developer
Posts: 2299
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by bill6502 »

@mattlach I can't speak to the the initial anecdotal comment about the "old method" of Schedules Direct
being discontinued (it's known as DataDirect). But DataDirect is no longer supported in v31 as you'll see in
the 2nd bullet in the Major Changes section of its Release Notes.

Some have reported having trouble converting to the new tv_grab_zz_sdjson or tv_grab_zz_sdjson_sqllite
XMLTV method. I agree that it can be tough. Just remember to do it as user mythtv, not your own user.

I suggest converting to it BEFORE upgrading to v31 as it will work on earlier versions of MythTV, at least v30.
Doing that eliminates a step required in the conversion.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

bill6502 wrote:
Mon Aug 08, 2022 1:08 am
@mattlach I can't speak to the the initial anecdotal comment about the "old method" of Schedules Direct
being discontinued (it's known as DataDirect). But DataDirect is no longer supported in v31 as you'll see in
the 2nd bullet in the Major Changes section of its Release Notes.

Some have reported having trouble converting to the new tv_grab_zz_sdjson or tv_grab_zz_sdjson_sqllite
XMLTV method. I agree that it can be tough. Just remember to do it as user mythtv, not your own user.

I suggest converting to it BEFORE upgrading to v31 as it will work on earlier versions of MythTV, at least v30.
Doing that eliminates a step required in the conversion.
Thank you for that suggestion. It seems like a very valuable one.
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

mtvf2894 wrote:
Sun Aug 07, 2022 10:18 am
Suggest reading the Major Changes for v31 and v32 (perhaps skim the other changes):

https://www.mythtv.org/wiki/Release_Notes_-_31
https://www.mythtv.org/wiki/Release_Notes_-_32

If you use hardware accelerated playback there may be issues. Weirdly for me v31 / 20.10 worked sort of okay but since going to v32 / 22.04 hardware accelerated playback has been very flaky. Also worth noting that Bookmarks have also changed https://github.com/MythTV/mythtv/commit/c6569589a8

I usually do a clean install of both the OS and MythTV but last time around I did a clean install of the OS and restored the MythTV database using a backup. This saved me from having to reconfigure MythTV which saved many hours. If you go this route make sure you check mythbackend.log to ensure the database schema updates have been successfully executed when you first run the backend.

I would use 22.04 why make future work for yourself (assuming it works with your containerization).
Appreciate the heads up, but in my particular application, I only use the backend. I don't use MythTV for playback at all, and hvaent since briefly testing it in ~2014. All of my frontends run Kodi/LibreElec/CoreElec (Depending on the device), and access the MythTV backend using the MythTV plugin, thus all of the hardware acceleration is handled by Kodi.
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by paulh »

bill6502 wrote:
Mon Aug 08, 2022 1:08 am
@mattlach I can't speak to the the initial anecdotal comment about the "old method" of Schedules Direct
being discontinued (it's known as DataDirect). But DataDirect is no longer supported in v31 as you'll see in
the 2nd bullet in the Major Changes section of its Release Notes.
A little history lesson for anyone interested ;)

Initially all projects using the old DataDirect service from SD was told they would have to switch to the new JSON service, at least that is how the initial reports sounded like. The reason given was that Gracenote that provided the DataDirect feed would be shutting down the feed in November 2014 but later SD decided to create there own drop-in replace feed served from there own servers.

In the mean time XMLTV had gained not one but two new grabbers that worked with the new JSON feed so the decision was made to only use XMLTV in mythfildatabase that way there was only one code path to maintain and any fixups would work for all grabbers from XMLTV. The JSON feed also provides data for more days and contains other information like season/episode numbers, show images etc. At least that was the theory :roll:

Hope you are all taking notes I will be asking question later :lol:

This is the original notice on the SD forum
https://forums.schedulesdirect.org/view ... f=3&t=2578
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

bill6502 wrote:
Mon Aug 08, 2022 1:08 am
@mattlach I can't speak to the the initial anecdotal comment about the "old method" of Schedules Direct
being discontinued (it's known as DataDirect). But DataDirect is no longer supported in v31 as you'll see in
the 2nd bullet in the Major Changes section of its Release Notes.

Some have reported having trouble converting to the new tv_grab_zz_sdjson or tv_grab_zz_sdjson_sqllite
XMLTV method. I agree that it can be tough. Just remember to do it as user mythtv, not your own user.

I suggest converting to it BEFORE upgrading to v31 as it will work on earlier versions of MythTV, at least v30.
Doing that eliminates a step required in the conversion.
Wow.

Well I feel silly now.

I was putting this off for so long as I had heard it described as difficult and would potentially break the system, but I decided I finally had to do with because Ubuntu 18.04 LTS goes EOL this month (unless you pay for ESM which I don't plan on doing)

Turns out it was just a drop down menu choice in the GUI config that took no time at all to get working.

Confirmed I am using tv_grab_zz_sdjson by doing a mythfilldatabase from the console and looking at the output.

That was easy...

So I snapshotted my system (in case things went awry), made the change in GUI, confirmed tv_grab_zz_sdjson was working in 30, then changed the apt sources from 30 bionic -> 31 bionic and ran apt update and ap dist-upgrade and now 31 is working just fine.

Next step is to move from bionic to focal. Here's hoping a straight up "do-release-upgrade" works without breaking anything.
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

mattlach wrote:
Mon Apr 17, 2023 3:28 am
bill6502 wrote:
Mon Aug 08, 2022 1:08 am
@mattlach I can't speak to the the initial anecdotal comment about the "old method" of Schedules Direct
being discontinued (it's known as DataDirect). But DataDirect is no longer supported in v31 as you'll see in
the 2nd bullet in the Major Changes section of its Release Notes.

Some have reported having trouble converting to the new tv_grab_zz_sdjson or tv_grab_zz_sdjson_sqllite
XMLTV method. I agree that it can be tough. Just remember to do it as user mythtv, not your own user.

I suggest converting to it BEFORE upgrading to v31 as it will work on earlier versions of MythTV, at least v30.
Doing that eliminates a step required in the conversion.
Wow.

Well I feel silly now.

I was putting this off for so long as I had heard it described as difficult and would potentially break the system, but I decided I finally had to do with because Ubuntu 18.04 LTS goes EOL this month (unless you pay for ESM which I don't plan on doing)

Turns out it was just a drop down menu choice in the GUI config that took no time at all to get working.

Confirmed I am using tv_grab_zz_sdjson by doing a mythfilldatabase from the console and looking at the output.

That was easy...

So I snapshotted my system (in case things went awry), made the change in GUI, confirmed tv_grab_zz_sdjson was working in 30, then changed the apt sources from 30 bionic -> 31 bionic and ran apt update and ap dist-upgrade and now 31 is working just fine.

Next step is to move from bionic to focal. Here's hoping a straight up "do-release-upgrade" works without breaking anything.
And just as easily, a "do-release-upgrade" moved us from 18.04 LTS to 20.04 LTS.

Then all I had to do was edit the files in /etc/apt/sources.list.d/ to rproperly reflect focal instead of bionic, and do an apt update and apt dist-upgrade and we were in business.

I feel like I should be knocking on wood or buying a lottery ticket or something, because all of this is just working.

I might just go for MythTV 32 and Ubuntu 22.04 while I am at it just so I don't have to do this again for a while.

Then again, not doing it for a while might be part of the problem, because then I forget what worked the last time :p
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

Hmm.

22.04 is trying to install a bunch of snap bullshit I don't want.

As luck would have it for some reason my server can't reach the snap store, which timed it out and allowed me to skip. I'm not quite sure why this happened, but I am glad it did. If it hadn't I would likely have rolled back to 20.04

This is reminiscent of installing Windows 10 and having it try to force me to create a Microsoft account, unless I disable all networking, so I can continue to use local accounts only.

Canonical and Microsoft are really behaving very similarly aren't they? Reprehensible.

It makes me want to investigate how well the MythTV backend runs on Debian and maybe go with that next time.

I don't ever want any binary blob distribution on my machine. No snaps, no Flatpak, no AppImage. Everything should be through apt/.deb

But of the three, I particularly don't want snaps.
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

And like that, we are on MythTV 33 on Ubuntu 22.04 LTS.

I expected this to go awry, and nothing did.

Maybe it IS time for a lottery ticket.

Only downside with the newer versions is that the backend configuration is surprisingly heavy weight graphically. On most local machines this might not be a problem, but my backend is a dedicated backend in a Linux container on my server in a rack. Only way to access it is to log in to LXDE via X2GO.

Everything works well, except the backend configuration which is SLOW in software only video output. Kind of wish the config were more basic.

(I mean, my preference would have been to have all the configuration in text files like I am used to for Linux stuff, but I am probably a minority there :p )
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
mattlach
Senior
Posts: 125
Joined: Wed Jun 11, 2014 2:52 am
United States of America

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by mattlach »

ack.

Turns out things did NOT work as well as I had hoped.

I could use some assistance.

The mythfilldatabase script runs, I have no channel data.

Code: Select all

myth@mythtv:~$ mythfilldatabase
2023-04-30 22:08:33.074280 C  mythfilldatabase version: fixes/33 [v33.1+fixes.202304101849.51cb10a678~ubuntu22.04.1] www.mythtv.org
2023-04-30 22:08:33.074289 C  Qt version: compile: 5.15.3, runtime: 5.15.3
2023-04-30 22:08:33.074342 I  Ubuntu 22.04.2 LTS (x86_64)
2023-04-30 22:08:33.074344 N  Enabled verbose msgs:  general
2023-04-30 22:08:33.074351 N  Setting Log Level to LOG_INFO
2023-04-30 22:08:33.084807 I  Added logging to the console
2023-04-30 22:08:33.085064 I  Setup Interrupt handler
2023-04-30 22:08:33.085069 I  Setup Terminated handler
2023-04-30 22:08:33.085072 I  Setup Segmentation fault handler
2023-04-30 22:08:33.085078 I  Setup Aborted handler
2023-04-30 22:08:33.085092 I  Setup Floating point exception handler
2023-04-30 22:08:33.085095 I  Setup Illegal instruction handler
2023-04-30 22:08:33.085097 I  Setup Bus error handler
2023-04-30 22:08:33.085107 I  Setup Real-time signal 0 handler
2023-04-30 22:08:33.085110 I  Setup Hangup handler
2023-04-30 22:08:33.085217 N  Using runtime prefix = /usr
2023-04-30 22:08:33.085219 N  Using configuration directory = /home/myth/.mythtv
2023-04-30 22:08:33.085262 I  Assumed character encoding: en_US.UTF-8
2023-04-30 22:08:33.085745 I  Empty LocalHostName. This is typical.
2023-04-30 22:08:33.085752 I  Using a profile name of: 'mythtv' (Usually the same as this host's name.)
2023-04-30 22:08:33.085869 I  Start up testing connections. DB localhost, BE , attempt 0, status dbAwake, Delay: 2000
2023-04-30 22:08:34.097715 N  Setting QT default locale to en_US
2023-04-30 22:08:34.097720 I  Current locale en_US
2023-04-30 22:08:34.097759 N  Reading locale defaults from /usr/share/mythtv//locales/en_us.xml
2023-04-30 22:08:34.120328 I  PowerDBus: Failed to delay sleep,shutdown: Permission denied
2023-04-30 22:08:34.141752 I  Power: On AC power
2023-04-30 22:08:34.141758 I  Power: Supported actions: None
2023-04-30 22:08:34.142682 I  Loading en_us translation for module mythfrontend
2023-04-30 22:08:34.143853 I  Current MythTV Schema Version (DBSchemaVer): 1376
2023-04-30 22:08:34.145832 I  MythCoreContext::ConnectCommandSocket(): Connecting to backend server: 10.0.1.17:6543 (try 1 of 1)
2023-04-30 22:08:34.146946 I  MythCoreContext::CheckProtoVersion(): Using protocol version 91 BuzzOff
2023-04-30 22:08:34.147132 I  Opening blocking connection to master backend
2023-04-30 22:08:34.713127 I  Updating source #1 (SchedulesDirect) with grabber tv_grab_zz_sdjson
2023-04-30 22:08:34.713794 I  Found 183 channels for source 1 which use grabber
2023-04-30 22:08:34.714053 I  Starting process manager
2023-04-30 22:08:34.714147 I  Starting process signal handler
2023-04-30 22:08:34.714165 I  Starting IO manager (read)
2023-04-30 22:08:34.715982 I  Starting IO manager (write)
2023-04-30 22:08:35.315176 I  Grabber has capabilities: baseline manualconfig preferredmethod 
2023-04-30 22:08:35.866066 I  Grabber prefers method: allatonce
2023-04-30 22:08:35.866869 I  XMLTV config file is: /home/myth/.mythtv/SchedulesDirect.xmltv
2023-04-30 22:08:36.467892 E  XMLTV grabber returned error code 1
2023-04-30 22:08:36.767713 E  Malformed XML file, missing </tv> element, at line 1, Premature end of document.
2023-04-30 22:08:37.396566 E  Failed to fetch some program info
2023-04-30 22:08:37.396591 I  Adjusting program database end times.
2023-04-30 22:08:37.397045 I      0 replacements made
2023-04-30 22:08:37.397046 I  Marking generic episodes.
2023-04-30 22:08:38.061898 I      Found 0
2023-04-30 22:08:38.061901 I  Extending non-unique programids with multiple parts.
2023-04-30 22:08:38.114856 I      Found 0
2023-04-30 22:08:38.114859 I  Fixing missing original airdates.
2023-04-30 22:08:38.367306 I      Found 0 with programids
2023-04-30 22:08:38.368269 I      Found 0 without programids
2023-04-30 22:08:38.368270 I  Marking repeats.
2023-04-30 22:08:38.459225 I      Found 0
2023-04-30 22:08:38.459229 I  Unmarking new episode rebroadcast repeats.
2023-04-30 22:08:39.007439 I      Found 0
2023-04-30 22:08:40.178553 I  Marking episode first showings.
2023-04-30 22:08:41.970979 I      Found 29883
2023-04-30 22:08:41.970983 I  Marking episode last showings.
2023-04-30 22:08:43.799256 I      Found 29696
2023-04-30 22:08:43.819492 I  
===============================================================
| Attempting to contact the master backend for rescheduling.  |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted.                            |
===============================================================
2023-04-30 22:08:43.820650 N  mythfilldatabase run complete.
2023-04-30 22:08:43.820696 I  PowerDBus: Closing interfaces
2023-04-30 22:08:43.820993 I  Waiting for threads to exit.
2023-04-30 22:08:43.822131 I  Exiting
I wind up with no data in any channel.

Appreciate any help ASAP!

--Matt
v33 backend in 22.04 LTS w. LXDE, in LXC on server w. 16C/32T Xeon E5-2650v2, 256GB RAM. 6C & 8GB assigned to container.
jarnos
Junior
Posts: 37
Joined: Sun Aug 11, 2019 8:17 am
Finland

Re: Suggestions and Tips for upgrading from v30 -> v32?

Post by jarnos »

mtvf2894 wrote:
Sun Aug 07, 2022 10:18 am
If you use hardware accelerated playback there may be issues. Weirdly for me v31 / 20.10 worked sort of okay but since going to v32 / 22.04 hardware accelerated playback has been very flaky.
What is your graphics adapter and driver? Maybe issues depend on those?
Post Reply