Schema upgrade hangs at Schema Version 1330

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

Post Reply
cwsolomon
Newcomer
Posts: 3
Joined: Sun Aug 21, 2016 1:09 am
Canada

Schema upgrade hangs at Schema Version 1330

Post by cwsolomon » Sun Aug 21, 2016 1:39 am

I am upgrading from Mythbuntu 14.04 to Mythbuntu 16.04 in a Virtualbox. The sudo do-release-upgrade fails in this configuration so I reverted to the VM pre-upgrade, and I installed a new VM with 16.04, then I took the database backup and restored the database in the 16.04 VM. The database restores successfully.

Code: Select all

mythserver@mythserver-VirtualBox:~$ service mythtv-backend stop
mythserver@mythserver-VirtualBox:~/.mythtv$ ./mythconverg_restore.pl --drop_database --create_database --verbose 

Configuring environment:
  -    username: mythserver
  -        HOME: /home/mythserver
  - MYTHCONFDIR: /home/mythserver/.mythtv

Parsing configuration files:
  - checking: /home/mythserver/.mythtv/config.xml
     parsing: /home/mythserver/.mythtv/config.xml
  - checking: /home/mythserver/.mythtv/backuprc
     parsing: /home/mythserver/.mythtv/backuprc

Applying command-line arguments.

Checking configuration.

No filename specified. Attempting to find the newest database backup.
Using database backup file:
/home/NAS-Backup/MythTV/mythconverg-1317-20160820190232.sql.gz

Database Information:
         DBHostName: localhost
             DBPort: 3306
         DBUserName: mythtv
         DBPassword: XXX
             DBName: mythconverg
        DBSchemaVer: 
  DBBackupDirectory: /home/NAS-Backup/MythTV
   DBBackupFilename: mythconverg-1317-20160820190232.sql.gz
      drop_database: yes
    create_database: yes

Executables:
       mysql_client: mysql
         uncompress: gzip -d

Miscellaneous:
    partial_restore: no
   restore_xmltvids: no
    change_hostname: no

Checking database.

Preparing initial database.
Dropping database.
Creating database.
Setting database character set.

Found 0 tables in the database.

Backup file is compressed.
 - Uncompressing backup file with IO::Uncompress::Gunzip.

Attempting to use supplied password for mysql command-line client.
Any [client] or [mysql] password specified in the MySQL options file will
take precedence.

Executing command:
'mysql' --defaults-extra-file='/tmp/H7oiCtfdgQ' --host='localhost' --port='3306' --user='mythtv' 'mythconverg'

mysql exited with status: 0

Restored 3700 of 3700 lines.

Successfully restored backup.

Found DBSchemaVer: 1317.
Setting database character set.
I start the mythtv-backend-setup and I get "Warning: MythTV wants to upgrade your database for the TV schema, from 1317 to 1344."
I click upgrade.

Code: Select all

Aug 20 21:27:54 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: E CoreContext dbutil.cpp:604 (DoBackup) Backing up database with script: '/usr/share/mythtv/mythconverg_backup.pl'
Aug 20 21:28:02 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbutil.cpp:625 (DoBackup) Database Backup complete.
Aug 20 21:28:02 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbutil.cpp:656 (DoBackup) Backed up database to file: '/tmp/mythconverg-1317-20160821012754.sql.gz'
Aug 20 21:28:09 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1318
Aug 20 21:28:09 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1319
Aug 20 21:28:09 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1320
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1321
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1322
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1323
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1324
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1325
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1326
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1327
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1328
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1329
Aug 20 21:28:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3820]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1330
mythserver@mythserver-VirtualBox:/var/log/mythtv$ 
It takes less than a second for each schema upgrade and it hangs at 1330. top shows mysqld running at 100% CPU. I have left it for over two hours with no change.

If I interrupt the process and run myth-backend setup again it will say "Warning: MythTV wants to upgrade your database for the TV schema, from 1330 to 1344." and it will still hang at the first step (after backud up database, before starting anything else.

Code: Select all

Aug 20 21:48:04 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3824]: E CoreContext dbutil.cpp:604 (DoBackup) Backing up database with script: '/usr/share/mythtv/mythconverg_backup.pl'
Aug 20 21:48:13 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3824]: C CoreContext dbutil.cpp:625 (DoBackup) Database Backup complete.
Aug 20 21:48:13 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3824]: C CoreContext dbutil.cpp:656 (DoBackup) Backed up database to file: '/tmp/mythconverg-1330-20160821014804.sql.gz'


I have read lots of forums and tried to run /usr/share/doc/mythtv-backend/contrib/maintenance/optimize_mythdb.pl before backing up the database and I have tried restoring the oldest backup in the autobackup history to see if that made a difference. Still the same happens.

I started the upgrade process because the frontends were complaining that the backend was too old.

I am an experienced LInux user and would appreciate any ideas that you may have.

User avatar
bill6502
Developer
Posts: 1814
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: Schema upgrade hangs at Schema Version 1330

Post by bill6502 » Sun Aug 21, 2016 4:08 am

Here's what's being done: https://code.mythtv.org/cgit/mythtv/tre ... .cpp#n2783

Note that in 1329, the same table was modified.

Anything in /var/log/mysql/errors.log?

cwsolomon
Newcomer
Posts: 3
Joined: Sun Aug 21, 2016 1:09 am
Canada

Re: Schema upgrade hangs at Schema Version 1330

Post by cwsolomon » Fri Aug 26, 2016 3:17 am

Thank you for your prompt response. Sorry I did not notice it until now. Nothing in var/log/errors. However, reading that code shows that the 1330 update requires an analysis on every video in the database, not just a modification of the tables; I have 4 TB of videos so it makes sense that this step takes a long time. I might just need to let it run for a very long time to see if it ever finishes. it looked strange that one change would take less than a second and the next would take hours, but that makes sense now....

i will post back to this list next week. I have other things to do this weekend.

cwsolomon
Newcomer
Posts: 3
Joined: Sun Aug 21, 2016 1:09 am
Canada

Re: Schema upgrade hangs at Schema Version 1330

Post by cwsolomon » Tue Aug 30, 2016 4:17 pm

I ran the upgrade and ignored it for a day, then checked the log. /var/log/mythtv/mythtvsetup.log

The schema change to schema 1330 took 12 hours and 13 minutes!..but eventually it worked. The whole thing appeared to hang while it was working on it. This was running in a virtualbox virtual machine with 8 GB of RAM (on a 32 GB RAMM Host with i7 3800 series cpu running at 3.3 GHz. This database has 3525 recordings.

Code: Select all

Aug 29 23:20:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbutil.cpp:656 (DoBackup) Backed up database to file: '/tmp/mythconverg-1317-20160830032002.sql.gz'
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1318
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1319
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1320
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1321
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1322
Aug 29 23:20:17 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1323
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1324
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1325
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1326
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1327
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1328
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1329
Aug 29 23:20:18 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1330
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:2846 (doUpgradeTVDatabaseSchema) Upgrading to MythTV schema version 1332
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1333
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1334
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1335
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1336
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1337
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1338
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1339
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1341
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1342
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:3224 (doUpgradeTVDatabaseSchema) Upgrading to MythTV schema version 1343
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: C CoreContext dbcheck.cpp:437 (performActualUpdate) Upgrading to MythTV schema version 1344
Aug 30 11:33:48 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: I CoreContext dbcheck.cpp:523 (UpgradeTVDatabaseSchema) Database schema upgrade complete.
Aug 30 11:34:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: N CoreContext mythmainwindow.cpp:2967 (IdleTimeout) Entering standby mode after 90 minutes of inactivity
Aug 30 11:34:10 mythserver-VirtualBox mythtv-setup.real: mythtv-setup[3922]: N CoreContext mythmainwindow.cpp:3017 (ExitStandby) Leaving standby mode
It might be wise to add a warning on the code for "upgrading to MythTV scheme version 1330 (this may take a long time)"

Thank you for sharing the code with me so that I understood what was happening.

Have a great day.

Post Reply