Page 1 of 1

mfdb fails when run from backend

Posted: Tue Jan 16, 2018 2:38 pm
by yajrendrag
just switched to xmltv with schedules direct.
using tv_grab_zz_sdjson.
installed to my normal user's home directory and symlinked to /home/mythtv/.mythtv and /home/mythtv/.xmltv
files in /home/jay/.mythtv and .xmltv are in group mythtv and mythtv user is in group jay and vice vera.

front end reports that mfdb is failing:

Code: Select all

"Last mythfilldatabase run started on Tue Jan 16, 1:00 AM and ended on Tue Jan 16, 1:03 AM. FAILED: XMLTV grabber returned error code 255."
it doesn't appear to me that new shows are added to the listings.

mythbackend log reports that mfdb finished successfully:

Code: Select all

"HouseKeeping housekeeper.cpp:153 (Run) HouseKeeperTask 'MythFillDB' Finished Successfully."
If i run mfdb interactively from the shell - with either user - jay or mythtv - mfdb works fine - finishes normally and updates the guide. so am thinking it's not a permission issue.

mythfilldatabase.log hasn't been updated since i switched over to xmltv - even when it runs successfully from interactive shell.

Prior to using xmltv, mfdb was working fine with data direct from schedules direct.

I can't find much of anything useful on this error code. any ideas on what to try or look for?

thx, Jay

Re: mfdb fails when run from backend

Posted: Tue Jan 16, 2018 3:02 pm
by bill6502
Hi,

If the log isn't updating, then check the program name in mythtv-setup.
It must be exactly: mythfilldatabase and not (for example) the full path
to that file. The help message should say that, it did, but I didn't check.
Check the permissions on the mythfilldatabase.log too.

Long ago I beta tested the JSON interface and it too created a log, or at least
it had the ability to do so. Maybe there's a -h option to to see if it still exists.

Re: mfdb fails when run from backend

Posted: Tue Jan 16, 2018 5:57 pm
by yajrendrag
It may have been that mythtv-setup had the full path to mfdb. i changed that and let mfdb run after mythtv-setup finished and it completed successfully. Not sure if that is equivalent environment as when the backend schedules it to run, but if so, then i should be good. otherwise, i will find out in the morning if it works.

thanks

Re: mfdb fails when run from backend

Posted: Wed Jan 17, 2018 3:56 pm
by yajrendrag
so changing the mythtv-setup program name to only mythfilldatabase (instead of full path) did fix the log issue, but did not fix the issue of mfdb populating updates to scheduling. Inspecting the logs, mythfilldatabase.log reports these three errors in the following sequence:

Code: Select all

...
E CoreContext filldata.cpp:440 (GrabData) FillData: XMLTV grabber returned error code 255
E CoreContext xmltvparser.cpp:658 (parseFile) Error in 1:1: unexpected end of file
I CoreContext filldata.cpp:320 (GrabDataFromFile) No programs found in data
E CoreContext main.cpp:432 (main) Failed to fetch some program info
...
To test this further, i built a cron job that simply runs mfdb and i did this twice - once in a shell script that runs under my normal users' cron and once under the mythtv user's cron. in each case the shell script is owned by the respective user. Both of these jobs completed successfully. ie, the above errors do not occur.

So, based on this, i am presuming this is not a permission or ownership issue with the way i set up xmltv (w/ the xmltv files in the user jay's .mythtv and .xmltv home directory and sym linked to respective entries in the mythtv user's home directory):

Code: Select all

in /home/mythtv/.mythtv:
lrwxrwxrwx 1 mythtv mythtv    29 Jan 11 13:32 cable.xmltv -> /home/jay/.mythtv/cable.xmltv

in home/mythtv/.xmltv:
lrwxrwxrwx 1 mythtv mythtv 40 Jan 11 13:30 tv_grab_zz_sdjson.cache -> /home/jay/.xmltv/tv_grab_zz_sdjson.cache
I am somewhat baffled as to why the mythtv-backend scheduled run of mfdb fails as above.

I suppose the easy "fix" is to simply cancel the backend's scheduled run and use my cron approach, but it just seems that something is wrong somewhere and i'd still like to find the issue.

let me know further thoughts on what else i might check.

thx, jay

Re: mfdb fails when run from backend

Posted: Wed Jan 17, 2018 4:39 pm
by MikeB2013
Can you post the full output of mythfilldatabase.log for the last run. Log snippets of just the error lines do not always show the whole picture, quite often the preceding lines are valuable.

Also can you post the contents of cable.xmltv, omitting the user id and password lines.

On my Xubuntu 16.04 LTS system I have the configuration files in /home/mythtv/.mythtv/ with the tv_grab_zz_sdjson cache path set to /home/mythtv/.xmltv/.... so I don't use any sym links and all is fine. If I need to run mythfilldatabase manually I switch to mythtv user and then run mythfilldatabase (note that mythfilldatabase.log does not get used in this case).

The other thing to check is the setting of videosource in mythconverg database, mine looks like this as I have 2 sources:

Code: Select all

mysql> select * from videosource;
+----------+----------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
| sourceid | name     | xmltvgrabber      | userid | freqtable | lineupid | password | useeit | configpath | dvb_nit_id |
+----------+----------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
|        1 | Freesat  | tv_grab_zz_sdjson |        | default   | NULL     | NULL     |      1 | NULL       |         -1 |
|        3 | Freeview | tv_grab_zz_sdjson |        | default   | NULL     | NULL     |      1 | NULL       |         -1 |
+----------+----------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
2 rows in set (0.00 sec)

Re: mfdb fails when run from backend

Posted: Wed Jan 17, 2018 6:47 pm
by yajrendrag
the two files are attached. (Note that the .txt extension is not there in the .mythv directory. i only added the .txt extension when i copied the file to my home directory to edit the user and password fields. for some reason, when i connected via smb to my myth box, i couldn't access the file to attach here without changing extension to .txt).

my video source shows up exactly like yours - except for the useit field, which on the surface seems like it's an obvious place to look:

mysql> select * from videosource;

Code: Select all

+----------+-------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
| sourceid | name  | xmltvgrabber      | userid | freqtable | lineupid | password | useeit | configpath | dvb_nit_id |
+----------+-------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
|        1 | cable | tv_grab_zz_sdjson |        | default   | NULL     | NULL     |      0 | NULL       |         -1 |
+----------+-------+-------------------+--------+-----------+----------+----------+--------+------------+------------+
I wonder how that got set wrong?

EDIT- i misread that - it's useEit not useit... probably not what i was thinking it was...

Re: mfdb fails when run from backend

Posted: Thu Jan 18, 2018 7:51 am
by MikeB2013
Thanks for the information.

The videosource settings are fine.

The mythfilldatabase log shows "...XMLTV config file is: /home/mythtv/.mythtv/cable.xmltv".

I think you have a permissions problem, when tv_grab_zz_sdjson is run by mythfilldatabase, data is cached according to the cache line in cable.xmltv. For normal running of mythfilldatabase, when invoked by mythbackend the user is mythtv, so user mythtv must be able to write to the cache directory.

From cable.xmltv the line "cache=/home/jay/.xmltv/tv_grab_zz_sdjson.cache" suggest changing jay to mythtv i.e.
"cache=/home/mythtv/.xmltv/tv_grab_zz_sdjson.cache"

Personally, I would also put cable.xmltv in /home/mythtv/.mythtv/ and avoid any sym links.

To run mythfilldatabase manually do "sudo su mythtv" in a terminal and then mythfilldatabase.

Mike

Re: mfdb fails when run from backend

Posted: Fri Jan 19, 2018 4:15 am
by yajrendrag
Thanks, I will try those changes
-jay

Re: mfdb fails when run from backend

Posted: Sat Jan 20, 2018 8:21 am
by yajrendrag
i removed my sym links and simply copied the files from user jay's home director to same locations in user mythtv's home directory. edited cable.xmltv to path of /home/mythtv/.mythtv/cable.xmltv per your suggestion.

fully expected it to work - it didn't. (but if it had worked, i couldn't have explained it since running it as a cron job under mythtv user works fine as does running it interactively from mythtv user).

Something is still broken, but can't seem to figure out what.

Re: mfdb fails when run from backend

Posted: Sat Jan 20, 2018 2:36 pm
by bill6502
I'd look at a fresh mfdb log. In the above, /home/mythtv/.mythtv/cable.xmltv
was able to be opened for reading, but: "Error in 1:1: unexpected end of file" fired.
E.g. the file was empty (1:1 = line:column of the failure.)

Need to know if the error has changed without the symlinks.

Re: mfdb fails when run from backend

Posted: Sat Jan 20, 2018 3:48 pm
by yajrendrag
Error is the same. i've attached the log.

another observation - maybe completely unrelated to this problem, dunno - but, since making the change to xmltv from data direct and since making the adjustment to mfdb command line in mythtv-setup, so that /var/log/mythtv/mythfilldatabase.log is written, i also get a copy of the log written to /home/mythtv. this didn't happen prior to the change from data direct.