Upgrading Ubuntu crashed MythTV

[white_haired_uncle]

Please change /etc/mysql/conf.d/mythtv.cnf:

Code: Select all

bind-address=*

to

Code: Select all

bind-address=127.0.0.1,192.168.1.27

and restart mysql

[bill6502]

@white_haired_uncle, if you haven't already spun up 24.04, I'm running it (with MythTV v35-Pre)
and it's working fine.

[white_haired_uncle]

I have, and it looks like a mysql/ubuntu/systemd (it's always systemd) bug.

I'll bet you're using localhost, and no bind-address, or a specific bind-address or list. Try any wildcard bind address (0.0.0.0, *, ::) and watch it break. Specifically, the bind-address=* that mbliss had from her old mythtv install, and that every mysql doc I can find says should make mysql listen on all addresses (instead of none, which is what it is doing for us).

An address list works for me (no wildcards).

[bill6502]

* is what I've been using. I have a comment in the .cnf file that I changed to * in 4/2022.
Server version: 8.0.39-0ubuntu0.24.04.2 is the MySQL version I'm on.

Do we know MySQL is in play, rather than MariaDB? This https://mariadb.com/kb/en/server-system ... nd_address
reads that * has the same meaning as in MySQL.

I am using localhost in config.xml so that a socket will be used rather then TCP.

[bill6502]

@mbliss, try this: sudo cat /var/log/mysql/error.log it may have some clues.
sudo zcat /var/log/mysql/error.log.?.gz would print all the earlier log contents (one
or more may have results from when the bind-address was *.

[white_haired_uncle]

Hmm, perhaps it's not mysql but netstat that's acting funny here.

Looking at this, I find it odd that I can connect to mysql using both 127.0.0.1 and 192.168.124.90 (both should fail):

Code: Select all

u2404$ netstat -an | grep 3306
tcp        0      0 127.0.0.1:33060         0.0.0.0:*               LISTEN     
tcp6       0      0 :::3306                 :::*                    LISTEN     

And yet:

Code: Select all

oot@u2404:/home/mythtv/.mythtv# netstat -an | grep 3306
tcp        0      0 127.0.0.1:33060         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:46232         127.0.0.1:3306          ESTABLISHED
tcp6       0      0 :::3306                 :::*                    LISTEN     
tcp6       0      0 127.0.0.1:3306          127.0.0.1:46232         ESTABLISHED

Code: Select all

root@u2404:/home/mythtv/.mythtv# netstat -an | grep 3306
tcp        0      0 127.0.0.1:33060         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:46232         127.0.0.1:3306          TIME_WAIT  
tcp        0      0 192.168.124.90:50276    192.168.124.90:3306     ESTABLISHED
tcp6       0      0 :::3306                 :::*                    LISTEN     
tcp6       0      0 192.168.124.90:3306     192.168.124.90:50276    ESTABLISHED

So, mysql isn't listening on port 3306 TCPv4, but you can connect to it. And that connection will be over TCPv4 when looked at from one perspective, but TCPv6 (using a v4 address!) from the other.

And it's not just netstat:

Code: Select all

root@u2404:/home/mythtv/.mythtv# ss -an6 | grep 3306
tcp   LISTEN 0      100                          *:3306                        *:*           
tcp   ESTAB  0      0      [::ffff:192.168.124.90]:3306  [::ffff:192.168.124.90]:36800       
root@u2404:/home/mythtv/.mythtv# ss -an4 | grep 3306
tcp   LISTEN    0      70                 127.0.0.1:33060        0.0.0.0:*          
tcp   TIME-WAIT 0      0                  127.0.0.1:60610      127.0.1.1:3306       
tcp   ESTAB     0      0             192.168.124.90:36800 192.168.124.90:3306       

Too weird. But, this means that looking at netstat (I) probably lead us down the wrong path, and mysql is probably working correctly and should connect.
Sorry about that.

[now on to figure out why my brand new v34 install has no tables in mythconverg]

[mbliss]

I'm a bit lost in the dust here, but I did change mythtv.cnf to use the ip list instead of the wildcard, but we still have no backend.
On the other hand, mysql.error.log contains this: [Warning] [MY-010058] [Server] Hostname 'Generosity' does not resolve to '192.168.1.27'.

[white_haired_uncle]

So, it looks like there's been some changes (not mythtv) that led me down a hole thinking mysql wasn't listening properly, but it probably is. You should be able to put bind-address=* back.

I would try connecting to mysql again, like these:

Code: Select all

$ mysql --user=mythtv --host=192.168.1.27 mythconverg -p
$ mysql --user=mythtv --host=localhost mythconverg -p
$ mysql --user=mythtv --host=127.0.0.1 mythconverg -p

You're looking for a couple things. One is if you can reach mysql at all (even if you get an error like "ERROR 1045 (28000): Access denied for user" that tells us you can make contact with it), the other is if you can authenticate (get something like a mysql> prompt). If you do get in, run:

Code: Select all

mysql> show tables;

and let us know if it finds anything.

I don't know when your last good database backup was, or if you have a copy saved somewhere off of this machine, but if you don't have one I'd make that your priority as soon as you can connect successfully to mysql. (You can also just stop mysql and tar up /var/lib/mysql as a failsafe).

The mysql.error.log is curious, particularly the capital letter (technically, that should not matter, but it's unusual). I would look at /etc/hosts, and

Code: Select all

$ ip addr
$ ip -r addr

for clues. But, it's a warning, so I wouldn't get too hung up on it.

[mbliss]

MySQL reports:

Code: Select all

mysql> show tables:
    ->

and

Code: Select all

dad@generosity:~$ cat /etc/hosts
127.0.0.1	localhost
127.0.1.1	Generosity

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

I don't understand "127.0.1.1 Generosity" at all as I had changed the hostname to all lowercase, and what's with 127.0.1.1?

There is also the issue with the backend log wherein we have

Code: Select all

░ A start job for unit mythtv-backend.service has begun execution.
░░ 
░░ The job identifier is 204.
Sep 18 08:29:40 generosity bash[1733]: /usr/share/mythbuntu/hdhomerun-discover.py:112: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
Sep 18 08:29:40 generosity bash[1733]:   start = datetime.utcnow()
Sep 18 08:29:41 generosity systemd[1]: mythtv-backend.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ An ExecStartPre= process belonging to unit mythtv-backend.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 2.

Is the "Invalid argument" related to the problem with finding the HDHomeRun, or is it something else? Doesn't seem like the deprecation warning would be fatal. Anyway, how would one find/fix either one? Doesn't seem like we're getting anywhere ...

[dnalorernst]

Looking at the include order of the mysql configuration

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

and the given grep output

$ grep -r bind /etc/mysql/
/etc/mysql/mysql.conf.d/mysqld.cnf:bind-address = 127.0.0.1
/etc/mysql/mysql.conf.d/mysqld.cnf:mysqlx-bind-address = 127.0.0.1
grep: /etc/mysql/debian.cnf: Permission denied
/etc/mysql/conf.d/mythtv.cnf:bind-address=*

then the bind-adress is set to 127.0.0.1
I suggest to move the last file 'mythtv.cnf' to

/etc/mysql/mysql.conf.d/

[white_haired_uncle]

Definitely check out what @dnalorernst wrote if you want to be able to use the IP (actually, fix it anyway). I'm guessing you were able to connect with localhost/127.0.0.1, but probably not 192.168.whatever. If that's the case, there's a good chance @dnalorernst's fix will allow mythbackend to connect to the database (last I knew you were using 192.168.whatever in config.xml - though at some point we may need to make sure you're using the right one -- should be able to tell from the mythbackend log).

Code: Select all

show tables;

<== that's a semicolon, not a colon

hostname and what's in /etc/hosts are two different things. Anything that starts with 127 is your local host.

I'm not sure what /usr/share/mythbuntu/hdhomerun-discover.py is, probably the workaround for systemd being unable to do the one thing it was supposed to do. I assumed this wasn't stopping mythbackend from starting since you were getting logs. You could always stop mythtv-backend in systemctl and just run it from the CLI while you're troubleshooting.

Looking at that script, it returns 2 when "more than 1 IP". It looks like it's coming from "hdhomerun_config discover" which you could run from the CLI to look at.

[bill6502]

@mbliss, here's a newer version of the HDHR program: https://pastebin.com/raw/iAF1VRLD
Do this: systemctl cat mythtv-backend and look at the ExecStartPre line. I run with the path
preceded with a dash character meaning that if the process fails, the backend will still start.
You can edit the setting with: sudo --preserve-env systemctl edit mythtv-backend.service

Run the HDHR program and paste the output here please. It sounds like you either have
more than one HDHR on your network and/or a response with an IPv4 AND IPv6 address
is coming back.

[mbliss]

Sorry, I give up. As far as I can tell, the upgrade torpedoed the database when it tried and failed to upgrade the format. phpMyAdmin can't find it either. The only backend log exists from the time of the upgrade and the backend hasn't run at all since then. Time to uninstall and start over. I sure miss LinHES, and sad that no one took over its development. Ubuntu has been a nightmare, but it doesn't appear that there is much support for any other platform. If this doesn't work, I'll reinstall v22 and try from there, refusing any upgrades at all! Thanks so much to all of you for all of your efforts, but there's a time to cut one's losses, and this is it.

[mbliss]

And it didn't work, remove and reinstall and the backend won't start, no access to the setupwizard. Guess I'll need to wipe the disk and start from a fresh install. I'm developing a real hatred for Ubuntu ....

[bill6502]

Would have been nice if you pasted systemctl cat mythtv-backend output. Did you
check the ExecStartPre I mentioned above?