[Solved] MythTV 29.1, mythfilldatabase problem

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

Post Reply
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

[Solved] MythTV 29.1, mythfilldatabase problem

Post by glen96 »

I installed mythtv version 29.1 by PPA.

Having completed the back end initial setup and in doing set "No Grabber" etc for shepherd.

Shepherd ran and filled shepherd output.xmltv with 7.3 MB of data that looked like the real deal.

After a rebooted I ran mythfilldatabase and it terminated without pulling in data.

The following is a copy of the terminal run:
2019-06-02 10:50:10.270074 C mythfilldatabase version: [v29.1]
2019-06-02 10:50:10.270086 C Qt version: compile: 5.9.5, runtime: 5.9.5
2019-06-02 10:50:10.270088 N Enabled verbose msgs: general
2019-06-02 10:50:10.270094 N Setting Log Level to LOG_INFO
2019-06-02 10:50:10.280488 N Only updating guide data, channel and icon updates will be ignored
2019-06-02 10:50:10.280596 I Added logging to the console
2019-06-02 10:50:10.280908 I Setup Interrupt handler
2019-06-02 10:50:10.280917 I Setup Terminated handler
2019-06-02 10:50:10.280922 I Setup Segmentation fault handler
2019-06-02 10:50:10.280928 I Setup Aborted handler
2019-06-02 10:50:10.280932 I Setup Bus error handler
2019-06-02 10:50:10.280938 I Setup Floating point exception handler
2019-06-02 10:50:10.280942 I Setup Illegal instruction handler
2019-06-02 10:50:10.280955 I Setup Real-time signal 0 handler
2019-06-02 10:50:10.280959 I Setup Hangup handler
2019-06-02 10:50:10.281050 N Using runtime prefix = /usr
2019-06-02 10:50:10.281052 N Using configuration directory = /home/glen/.mythtv
2019-06-02 10:50:10.281105 I Assumed character encoding: en_AU.UTF-8
2019-06-02 10:50:10.281345 N Empty LocalHostName.
2019-06-02 10:50:10.281351 I Using localhost value of myth-HP-BE2
2019-06-02 10:50:10.281391 I Start up testing connections. DB localhost, BE , attempt 0, status start
2019-06-02 10:50:11.797487 N Setting QT default locale to en_AU
2019-06-02 10:50:11.797494 I Current locale en_AU
2019-06-02 10:50:11.797516 E No locale defaults file for en_AU, skipping
2019-06-02 10:50:11.797915 I Loading en_us translation for module mythfrontend
2019-06-02 10:50:11.798602 I Current MythTV Schema Version (DBSchemaVer): 1348
2019-06-02 10:50:11.799826 I MythCoreContext ConnectCommandSocket(): Connecting to backend server: 192.168.1.13:6543 (try 1 of 1)
2019-06-02 10:50:11.800617 I MythCoreContext::CheckProtoVersion(): Using protocol version 91 BuzzOff
2019-06-02 10:50:11.800787 I Opening blocking connection to master backend
2019-06-02 10:50:11.802451 I Source 1 configured with no grabber. Nothing to do.<------------
2019-06-02 10:50:11.803561 N Data fetching complete.
2019-06-02 10:50:11.803567 I Adjusting program database end times.
2019-06-02 10:50:11.803696 I 0 replacements made
2019-06-02 10:50:11.803698 I Marking generic episodes.
2019-06-02 10:50:11.807103 I Found 0
2019-06-02 10:50:11.807106 I Extending non-unique programids with multiple parts.
2019-06-02 10:50:11.808090 I Found 0
2019-06-02 10:50:11.808093 I Fixing missing original airdates.
2019-06-02 10:50:11.809694 I Found 0 with programids
2019-06-02 10:50:11.813770 I Found 0 without programids
2019-06-02 10:50:11.813772 I Marking repeats.
2019-06-02 10:50:11.816422 I Found 0
2019-06-02 10:50:11.816424 I Unmarking new episode rebroadcast repeats.
2019-06-02 10:50:11.816858 I Found 0
2019-06-02 10:50:11.823784 I Marking episode first showings.
2019-06-02 10:50:11.837100 I Found 391
2019-06-02 10:50:11.837102 I Marking episode last showings.
2019-06-02 10:50:11.850416 I Found 385
2019-06-02 10:50:11.850882 I
===============================================================
| Attempting to contact the master backend for rescheduling. |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted. |
===============================================================
2019-06-02 10:50:11.851389 N mythfilldatabase run complete.
2019-06-02 10:50:11.851412 I Waiting for threads to exit.
The line 2/3 down says:
2019-06-02 10:50:11.802451 I Source 1 configured with no grabber. Nothing to do.
I don't know but I thought this may indicate a bug with mythfilldatabase. If not a bug then I don't know what to try next.

Hope you can help
Cheers
Glen
User avatar
bill6502
Developer
Posts: 2307
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: MythTV 29.1, mythfilldatabase problem

Post by bill6502 »

Hi,

If the grabber is set to: /bin/true or none or is just blank, then the Information
message you mentioned fires. Not a bug.

I'm not a Shepard user, but some searching suggests their grabber gets run and then
mythfilldatabase is run with the --file option. For example, on this page about 1/2 way down:

https://blog.christophersmart.com/2008/ ... ide-down/ l

You'll see that at least on that page, cron is used. It's pretty old and has some broken links.

If you wanted, a systemd .timer and .service could be created rather than cron or anacron.

You can test with the 7.3MB file you mentioned above using mythfilldatabase --file shepard_file_name

There is a regional forum here for Australia. If some kind Shepard user doesn't
respond here, you could try there.

Good luck.
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

Re: MythTV 29.1, mythfilldatabase problem

Post by glen96 »

Hi bill6502
Thank you.

The reference you refereed to dated 2008 was relevant when I built my first box in 2013. That box stopped getting program data about a month ago.

I am building the new box with reference to https://github.com/ShephedProject/sheph ... pendencies (2018), which requires the listings grabber to be set to No Grabber, I could not find any reference in Configuring MythTV which outlines a procedure from MythTV point of view.

Shepherd gathers 7.3MB of data but mythfilldatabase does not work with the data. I am missing some thing but what. I thought mythfilldatabase job was to import the data to the DB.

Thanks for any help you can offer.
glen
User avatar
bill6502
Developer
Posts: 2307
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: MythTV 29.1, mythfilldatabase problem

Post by bill6502 »

Good link. I looked at their source, but I don't do Perl.

It still appears to me that the grabbber is run out of cron. The output is then processed by mythfilldatabase
like this: mythfilldatabase --only-update-guide --file --sourceid 1 --xmlfile $output_filename". Where
$output_filename is = $CWD/output.xmltv.

Have you looked at your own crontab, e.g. crontab -l.

The No Grabber makes sense to me now. Not for getting the data, but for processing what
crontab's entry has inserted into MythTV's DB.

Seems to me that they don't parse the current versions correctly and if they can't find it, set it to 0.99.
I <think> that's OK though. Looks like when the grabber is run it should output the version it detected.
As long as it's over 0.29 it should pick the correct mythfilldatabase command line.
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

Re: MythTV 29.1, mythfilldatabase problem

Post by glen96 »

I have tried to progress but I am not getting anywhere, just totally confused.

I would like to reset the system back to the start of the Scan for Channels. There are so many processes with side affects that I do not understand. Is it possible to reset or (horror) should I reset by reloading ubuntu?

I cant continue the way things are, shepherd gathers the data but mythfilldatabase does not work with the data during the daily update.
User avatar
dizygotheca
Developer
Posts: 267
Joined: Wed Sep 03, 2014 9:02 am
Great Britain

Re: MythTV 29.1, mythfilldatabase problem

Post by dizygotheca »

Not a Shepherd user but reinstalling is a bit drastic. if Myth & Shepherd are working otherwise then you just need a simple configuration fix to tie them together.

Setting the grabber to "No grabber" and removing any XMLTV related files from ~glen/.mythtv should revert you back to a clean state.

The standard/Myth pattern for XMLTV is that mythbackend runs mythfilldatabase daily as the backend user (mythtv). This means;
  • "Automatically run mythfilldatabase" is on,
  • The XMLTV configuration files should be accessible to the mythtv user in ~mythtv/.mythtv.
  • mythtv-setup must be run as backend user when configuring the grabber so that the configuration files get written to the right place
Shepherd refer to this as "The old method" (some of their trouble-shooting FAQ *may* be out-of-date). And Myth documentation/help may not be applicable

The 'new method' appears to be that the shepherd 'grabber' is actually a script that replaces the backend role.
It seems to invoke itself daily via crontab, gathers data and then invokes mythfilldatabase as user glen to update Myth. This means;
  • mythbackend never runs mythfilldatabase at all ("Automatically run mythfilldatabase" should be off). If it tries, it will fail as XMLTV is unconfigured for the mythtv user
  • The XMLTV configuration files should be accessible to the glen user in ~glen/.mythtv.
  • Running mythtv-setup as glen user is probably ok for you.
Running CLI mythfilldatabase *may* not be valid. The shepherd log will hopefully show exactly how it's invoking mythfilldatabase. It could be adding options that you are missing off.

Code: Select all

tv_grab_au --refill-mythtv
is probably a better way of testing and may produce informative errors.

More likely is that

Code: Select all

~/.shepherd/shepherd --configure-mythtv
didn't work properly. Try it again (noting the sudo issue) and check for errors in any shepherd logs.
I'd expect it to create a configuration file something like ~glen/.mythtv/tv_grab_au.xmltv and mythtv-setup should then show a configured tv_grab_au grabber.
Possible failures are sudo, missing perl dependencies, missing xmltv or basic permissions.
The Google group may be a better place to follow-up those issues.
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

Re: MythTV 29.1, mythfilldatabase problem

Post by glen96 »

Such an in depth reply thank you dizygotheca.

I got stuck into it today:
Deleted glen/shepherd and glen/shepherd.
Reset: no grabber, deleted all scan data, re scand channels, edited channel numbers to match channel names.
Exited setup BE.
At this point ran mythfilldatabase. I don't think wiki says to do this but my notes from 2013 said do it to initialize the DB.
It may have been unnecessary but it appears to have done no harm. (this may be a wiki oversight)
Downloaded shepherd and ran it.
This time all the shepherd config questions 4..11 where asked and answered as expected.
When shepherd finished I rebooted. I don't think this is in the wiki, but my notes from 2013 called for it
Following reboot ran shepherd and that lasted about 2 hours.
Started FE and ............ i NOW HAVE 8 DAYS OF DATA !!!!!!!!!!!

The daily update should happen automatically at about 1500 local time tomorrow.

Thank you both bill6502 & dizygotheca for the time, effort and knowledge you give to MythTV and the community.

Cheers
Glen
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

Re: MythTV 29.1, mythfilldatabase problem

Post by glen96 »

Slept well last night but when I booted this morning, shock horror the BE could not start the DB.

Spent today trying to solve the problem with reference to viewtopic.php?t=1952 and viewtopic.php?t=2788 plus some other reads.

Checked both versions of mythtv config.xml they looked the same. Tries some of the suggested terminal commands but no positive results.

Installed MySQL Workbench and defined a connection with the config.xml data and it worked perfectly, I viewed the content of tables channel and program, all the data was there.

I suspect this is probably an easy to fix problem, it seems I have a lot to learn.

Glen
glen96
Newcomer
Posts: 11
Joined: Thu May 09, 2019 12:08 am
Australia

[SOLVED] Re: MythTV 29.1, mythfilldatabase problem

Post by glen96 »

I have just realized this is now off topic target. This will be the last post for this topic.

With reference to viewtopic.php?t=2788 I tried

Code: Select all

ln -s -f /etc/mythtv/config.xml ~/.mythtv/config.xml
this appears to solve my problem.

File property ~/.mythtv/config.xml > kind = "link to /etc/mythtv/config.xml".

I then tried to run the FE, all worked normally, looked around and then exited FE.

The File property had changed ~/.mythtv/config.xml > kind = "XML document", but all subsequent FE tests passed ok .

Perhaps this side affect of running the FE may be of interest to the developers.
Post Reply