mythfilldatabase sdjson

[wesnewell]

Runs fine from terminal as user, but get errors when run from mythtv.
I tried everything I know but it just won't run from mythtv backend. Tried copying files to mthtv dir. Still get errors. Tried running mythv-set from mythtv user account after creating a passwd for it. No go.
Finally created a crontab job to run a simple shell script each day from my home dir and that seems to work ok, but that doesn't seem like the proper way to do it. Anyone got any ideas?
Errors in log show up as error 1, and error 255, depending on how I configure the backend setup.
Aug 8 08:21:32 mythfe0 mythfilldatabase: mythfilldatabase[1954]: E CoreContext filldata.cpp:441 (GrabData) FillData: XMLTV grabber returned error code 1
Aug 8 08:21:32 mythfe0 mythfilldatabase: mythfilldatabase[1954]: E CoreContext xmltvparser.cpp:670 (parseFile) Error in 1:1: unexpected end of file
Aug 8 08:21:32 mythfe0 mythfilldatabase: mythfilldatabase[1954]: I CoreContext filldata.cpp:321 (GrabDataFromFile) No programs found in data.
...
Aug 9 10:01:48 mythfe0 mythfilldatabase: mythfilldatabase[19168]: E CoreContext filldata.cpp:441 (GrabData) FillData: XMLTV grabber returned error code 255
Aug 9 10:01:48 mythfe0 mythfilldatabase: mythfilldatabase[19168]: E CoreContext xmltvparser.cpp:670 (parseFile) Error in 1:1: unexpected end of file
Aug 9 10:01:49 mythfe0 mythfilldatabase: mythfilldatabase[19168]: I CoreContext filldata.cpp:321 (GrabDataFromFile) No programs found in data.

Other files.
wes@mythfe0:~$ cat ~/.mythtv/sd.xmltv
cache=/home/wes/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=default
previously-shown-format=date
username=xxxxxxx
password=xxxxxxxxxx
mode=lineup
lineup=USA-OTA-75098

wes@mythfe0:~$ cat /home/mythtv/.mythtv/sd.xmltv
cache=/home/mythtv/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=default
previously-shown-format=date
username=xxxxxxxx
password=xxxxxx
mode=lineup
lineup!USA-OTA-75098

wes@mythfe0:~$ ll ~/.mythtv/sd*
-rw-rw-r-- 1 wes wes 180 Aug 8 10:00 /home/wes/.mythtv/sd.xmltv
wes@mythfe0:~$ ll ~/.xmltv/*
-rw-rw-rw- 1 wes mythtv 52480352 Aug 11 00:18 /home/wes/.xmltv/tv_grab_zz_sdjson.cache
wes@mythfe0:~$ ll /home/mythtv/.mythtv/sd*
-rw-rw-r-- 1 mythtv mythtv 183 Aug 10 06:44 /home/mythtv/.mythtv/sd.xmltv
wes@mythfe0:~$ ll /home/mythtv/.xmltv/*
-rw-rw-rw- 1 mythtv mythtv 52319942 Aug 7 07:46 /home/mythtv/.xmltv/tv_grab_zz_sdjson.cache
wes@mythfe0:~$

wes@mythfe0:~$ crontab -l
# Edit this file to introduce tasks to be run by cron.
# m h dom mon dow command
18 10 * * * /home/wes/mfdb

wes@mythfe0:~$ cat mfdb
#!/bin/bash
mythfilldatabase

[MikeB2013]

Looks like a permissions issue.

Assuming a packaged version of mythtv e.g. on Ubuntu, mythfilldatabase runs under user "mythtv" when automatically run. If you want to run mythfilldatabase from a terminal session, you need to switch to mythtv user by doing sudo su mythtv in the terminal session.

Make sure no other versions of sd.xmltv files exist, except in /home/mythtv/.mythtv/

Your sd.xmltv file needs to be in /home/mythtv/.mythtv/ with the cache= line being cache=/home/mythtv/.xmltv/tv_grab_zz_sdjson.cache

Ownership of sd.xmltv and files in /home/mythtv/.xmltv/ should be mythtv:mythtv

Note I have never added a password to mythtv user, the only thing I have done is to add a shell to mythtv user which makes terminal usage a little easier (you get things like auto-complete) by doing sudo chsh -s /bin/bash mythtv

Mike

[wesnewell]

Looks like a permissions issue.
Ownership of sd.xmltv and files in /home/mythtv/.xmltv/ should be mythtv:mythtv

If you note in my first post, they are owned by mythtv and also in mythtv group with r/w permissions set.

This was originally a mythbuntu 16.04 install that I did a release upgrade to 18.04lts a while back when 18.04 first came out. Everything works properly using SD data direct. This problem only started when I swicthed to sdjson data about a week ago. There was no mythtv user you could log in as before I added the password for user mythtv. I think at this point, it may be easier to just re-install everything with 18.04.1 LTS, but I'm trying to avoid that. The crontab job does work and guide data does get updated so maybe I'll just leave it that way.

[MikeB2013]

Just noticed in
wes@mythfe0:~$ cat /home/mythtv/.mythtv/sd.xmltv
cache=/home/mythtv/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=default
previously-shown-format=date
username=xxxxxxxx
password=xxxxxx
mode=lineup
lineup!USA-OTA-75098

The lineup is disabled, the ! should be =

Your other files has the lineup enabled:
Other files.
wes@mythfe0:~$ cat ~/.mythtv/sd.xmltv
cache=/home/wes/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=default
previously-shown-format=date
username=xxxxxxx
password=xxxxxxxxxx
mode=lineup
lineup=USA-OTA-75098

[wesnewell]

Good catch. Don't know how that happened. Just ran mythfilldatabase as user mythtv without errors. It didn't insert any new data but I'll assume for now that's because I ran a crontab job as wes earlier today. Will delete crontab today and see if that fixed it tomorrow when it runs again.
Thank you very much.

Yep. ran like a charm today. thanks again.

[dlasher]

I wanted to add to this thread, since this seemed to the closest to what I was experiencing, and how I finally fixed it.

mythtv 0.31
schedulesdirect

I recently upgraded to 0.31, which meant my guide-data setups didn't work any more. Lots of searching, followed all the online guides, and in spite of all the XMLTV setup being correct for tv_grab_zz_sdjson_sqlite (user permissions, cache file location, etc), any time I'd run Mythfilldatabase, I'd get "No programs found in data"

Official Guide:
* https://www.mythtv.org/wiki/XMLTV#Setup ... _zz_sdjson

and also checked these threads:
* viewtopic.php?t=3942#p19227
* viewtopic.php?f=36&t=4760&p=23375
* viewtopic.php?f=36&t=3480&start=15#p19508
* viewtopic.php?t=1957

So what fixed it?

Instead of using "Multinational (Schedules Direct JSON web services with SQLite DB)" as my listing source, I set "Listings Grabber" to "Schedules Direct - JSON API (XMLTV)"

Credit to Wes:

cache=/home/mythtv/.xmltv/tv_grab_zz_sdjson.cache
channel-id-format=default
previously-shown-format=date
username=your sd username
password=your sd password
mode=lineup
lineup=USA-OTA-76078

Bingo - mfdb runs, staggering amounts of data, channels I don't want (different problem) but solved. I have neither the time or the energy to figure out why one grabber works, and the other doesn't, I'm just happy to have guide data again.


FAILED RUN:

Code: Select all

2022-05-04 20:02:13.795650 C  mythfilldatabase version: fixes/31 [v31.0+fixes.202203040618.7e4ce1ba98~ubuntu20.04.1]
2022-05-04 20:02:13.795678 C  Qt version: compile: 5.12.8, runtime: 5.12.8
2022-05-04 20:02:13.795712 I  Ubuntu 20.04.4 LTS (x86_64)
2022-05-04 20:02:13.807015 N  Using configuration directory = /home/mythtv/.mythtv
2022-05-04 20:02:15.291690 I  Updating source #3 (sd.comcast.XML) with grabber tv_grab_zz_sdjson_sqlite
2022-05-04 20:02:15.292490 I  Found 378 channels for source 3 which use grabber
2022-05-04 20:02:16.898828 I  XMLTV config file is: /home/mythtv/.mythtv/sd.comcast.XML.xmltv
2022-05-04 20:02:18.981910 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.001601 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.021059 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.040566 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.059761 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.079322 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:02:19.081219 I  No programs found in data.
2022-05-04 20:02:19.502257 N  Data fetching complete.
2022-05-04 20:02:19.502285 I  Adjusting program database end times.
2022-05-04 20:02:19.502955 I      0 replacements made
2022-05-04 20:02:19.502959 I  Marking generic episodes.
2022-05-04 20:02:21.199105 I      Found 0
2022-05-04 20:02:21.199116 I  Extending non-unique programids with multiple parts.
2022-05-04 20:02:21.390704 I      Found 0
2022-05-04 20:02:21.390709 I  Fixing missing original airdates.
2022-05-04 20:02:22.117468 I      Found 0 with programids
2022-05-04 20:02:22.119593 I      Found 0 without programids
2022-05-04 20:02:22.119604 I  Marking repeats.
2022-05-04 20:02:22.200904 I      Found 0
2022-05-04 20:02:22.200916 I  Unmarking new episode rebroadcast repeats.
2022-05-04 20:02:23.577694 I      Found 0
2022-05-04 20:02:26.115446 I  Marking episode first showings.
2022-05-04 20:02:28.826059 I      Found 40793
2022-05-04 20:02:28.826077 I  Marking episode last showings.
2022-05-04 20:02:31.538256 I      Found 40049
2022-05-04 20:02:31.577227 I  

SUCCESSFUL RUN:

Code: Select all

2022-05-04 20:04:40.318172 C  mythfilldatabase version: fixes/31 [v31.0+fixes.202203040618.7e4ce1ba98~ubuntu20.04.1]
2022-05-04 20:04:40.318226 I  Ubuntu 20.04.4 LTS (x86_64)
2022-05-04 20:04:40.329453 N  Using configuration directory = /home/mythtv/.mythtv
2022-05-04 20:04:41.811906 I  Updating source #3 (sd.comcast.XML) with grabber tv_grab_zz_sdjson
2022-05-04 20:04:41.812671 I  Found 378 channels for source 3 which use grabber
2022-05-04 20:04:41.813270 I  Starting process manager
2022-05-04 20:04:41.813249 I  Starting IO manager (read)
2022-05-04 20:04:41.813340 I  Starting process signal handler
2022-05-04 20:04:41.813377 I  Starting IO manager (write)
2022-05-04 20:04:42.415239 I  Grabber has capabilities: baseline manualconfig preferredmethod 
2022-05-04 20:04:43.016631 I  Grabber prefers method: allatonce
2022-05-04 20:04:43.017925 I  XMLTV config file is: /home/mythtv/.mythtv/sd.comcast.XML.xmltv
2022-05-04 20:12:55.263527 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.282968 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.302429 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.321798 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.342444 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.361790 I  Cardutil: HDHomeRun Cablecard Present.
2022-05-04 20:12:55.363574 I  New static DB connectionChannelCon
.... still going ....

As a long time user of mythtv, long time supporter of schedules direct - this is still far more painful than it needs to be. FAR more.. hoping something as integral as guide data gets easier to digest in later versions.