[Solved]xmltv upgrade for Sched Dir fails to provide grabber

[keepitsimpleengineer]

Updating mythbackend to use Schedules Direct sqlite but now Video Sources only list only contains "Transmitted guide only (EIT)" and "no grabber" ~ the sqlite grabber is missing

xmltv has been upgraded to 0.6.3-1 from 0.6.1-2 (on Manjaro from Archlinux AUR) and the line-up set successfully from the command line.

On Manjaro the mythtv home is at ~/.mythtv. After upgrade of xmltv the contents of ~/.xmltv were copied to ~/.mythtv

Code: Select all

ljohnson@KISE-055 ~]$ ls -lat .xmltv
total 27140
drwxr-xr-x  2 ljohnson ljohnson     4096 Oct 15 13:27 .
-rw-r--r--  1 ljohnson ljohnson 16470016 Oct 15 13:27 SchedulesDirect.DB
-rw-r--r--  1 ljohnson ljohnson      144 Oct 15 13:27 tv_grab_zz_sdjson_sqlite.conf.TMP
drwx------ 26 ljohnson ljohnson     4096 Oct 15 08:48 ..
-rw-r--r--  1 ljohnson ljohnson 11298459 Sep 30 09:06 tv_grab_zz_sdjson.cache
-rw-r--r--  1 ljohnson ljohnson      124 Apr 29 15:41 tv_grab_zz_sdjson_sqlite.conf
[ljohnson@KISE-055 ~]$ ls -lat .mythtv
total 20760
drwxr-xr-x 10 ljohnson ljohnson     4096 Oct 15 13:28 cache
drwxr-xr-x  4 ljohnson ljohnson     4096 Oct 15 12:09 themes
drwxr-xr-x  4 ljohnson ljohnson     4096 Oct 15 12:09 tmp
drwxr-xr-x 10 ljohnson ljohnson     4096 Oct 15 10:35 .
-rw-r--r--  1 ljohnson ljohnson      144 Oct 15 10:35 SD.xmltv
-rw-r--r--  1 ljohnson ljohnson      144 Oct 15 10:28 Antenna.xmltv.TMP
-rw-r--r--  1 ljohnson ljohnson      144 Oct 15 10:27 Antenna.xmltv
-rw-r--r--  1 ljohnson ljohnson 11298459 Oct 15 10:16 tv_grab_zz_sdjson.cache
-rw-r--r--  1 ljohnson ljohnson      124 Oct 15 10:16 tv_grab_zz_sdjson_sqlite.conf
-rw-r--r--  1 ljohnson ljohnson  9883648 Oct 15 10:16 SchedulesDirect.DB
drwx------ 26 ljohnson ljohnson     4096 Oct 15 08:48 ..
drwxr-xr-x  2 ljohnson ljohnson     4096 May 15 06:28 channels
-rw-r--r--  1 ljohnson ljohnson     2321 May  3 09:50 HD_Antenna.xmltv
-rw-r--r--  1 ljohnson ljohnson     2321 Apr 29 16:26 HD_antenna.xmltv
-rw-r--r--  1 ljohnson ljohnson     2321 Apr 29 16:12 HD_antenna.xmltv.old
drwxr-xr-x  2 root     root         4096 Feb 27  2020 python-scripts
-rw-r--r--  1 ljohnson ljohnson      626 Jun 16  2019 config.xml
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 HardwareProfile
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 MythBrowser
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 3rdParty

"Antenna.xmltv" was to be the new Video Source name

The documentation suggests the xmltv install should set up the sqlite grabber but this is not happening and I am at a total loss

Code: Select all

# pacman -Qii perl
Name            : perl
Version         : 5.32.0-3
Description     : A highly capable, feature-rich programming language
Architecture    : x86_64
URL             : https://www.perl.org
⋯ ⋯
Build Date      : Thu 17 Sep 2020 02:12:52 PM PDT
Install Date    : Thu 01 Oct 2020 07:56:16 AM PDT
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature
Backup Files    :
⋯ ⋯
# pacman -Qii xmltv
Name            : xmltv
Version         : 0.6.3-1
Description     : Set of utilities to download tv listings and format them in xml
Architecture    : any
URL             : http://xmltv.org/wiki/
⋯ ⋯
Build Date      : Thu 15 Oct 2020 12:36:27 PM PDT
Install Date    : Thu 15 Oct 2020 12:38:05 PM PDT
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : None
Backup Files    :

[bill6502]

tv_find_grabbers will print a list of grabbers. But, if that's not there, it
suggests that the xmltv package wasn't installed. I don't speak Archlinux AUR
though. I see above it was upgraded though. My grabbers are in /usr/bin.

I chose to use the grabber from github mentioned here:
https://www.mythtv.org/wiki/XMLTV#Setup ... son_sqlite

You computer/your choice, but the Wiki wants you to configure as user mythtv
and when mythfilldatabase is run by the backend, it'll find the proper files.
When run as a user (yourself), you can sudo to mythtv. I made an alias
that does this: "sudo --login --user=mythtv mythfilldatabase --verbose=xmltv"

There some ascii art here: viewtopic.php?f=36&t=3480&start=15#p19508

[keepitsimpleengineer]

OK ~ so I found and uninstalled 'tv_grab_zz_sdjson_sqlite-git' from 2018 in AUR, then reinstalled xmltv (AUR)

After the reinstall I found 'tv_find_grabbers' & 'tv_grab_zz_sdjson_sqlite' in '/usr/bin/vendor_perl/'

Running '[/usr/bin/vendor_perl/]./tv_find_grabbers" executes but no output.
(When I ran it prior to deleting 'tv_grab_zz_sdjson_sqlite-git' it produced the result

Code: Select all

$ ./tv_find_grabbers
/usr/bin/tv_grab_zz_sdjson_sqlite|Multinational (Schedules Direct JSON web services with SQLite DB)

Which suggests the install of xmltv(AUR) puts them somewhere other than needed ...?

I presume 'tv_find_grabbers' is run by myth-setup and that why they are not showing up there..?

[paulh]

Try running the tv_grab_zz_sdjson_sqlite command directly to see what happens. My guess is you are missing some dependency and it's failing to run. Try something like

Code: Select all

tv_grab_zz_sdjson_sqlite --description

or to see a full list of parameters you can try

Code: Select all

tv_grab_zz_sdjson_sqlite --help

Is that the only grabber you see or are there more tv_grab_* scripts there also?
On some packaging systems you can choose what grabbers get installed by setting some environment variable or changing some config file etc. I don't know whether Arch AUR does something like that or not though.

[keepitsimpleengineer]

Code: Select all

[ljohnson@KISE-055 vendor_perl]$ pwd
/usr/bin/vendor_perl
[ljohnson@KISE-055 vendor_perl]$ ls
binhex.pl    GET           mimetype                 tv_check            tv_grab_fi_sv       tv_grab_zz_sdjson           tv_validate_file
config_data  HEAD          package-stash-conflicts  tv_count            tv_grab_fr          tv_grab_zz_sdjson_sqlite    tv_validate_grabber
crc32        htmltree      POST                     tv_extractinfo_ar   tv_grab_huro        tv_grep                     widget
dbilogstrip  imgsize       ptked                    tv_extractinfo_en   tv_grab_is          tv_imdb                     xml_grep
dbiprof      json_xs       ptksh                    tv_find_grabbers    tv_grab_it          tv_merge                    xml_merge
dbiproxy     lwp-download  SOAPsh.pl                tv_grab_ar          tv_grab_na_dd       tv_remove_some_overlapping  xml_pp
debinhex.pl  lwp-dump      stubmaker.pl             tv_grab_combiner    tv_grab_na_dtv      tv_sort                     xml_spellcheck
dm_date      lwp-mirror    tkjpeg                   tv_grab_dk_dr       tv_grab_na_tvmedia  tv_split                    xml_split
dm_zdump     lwp-request   tv_augment               tv_grab_eu_epgdata  tv_grab_tr          tv_to_latex                 xpath
findrule     mech-dump     tv_augment_tz            tv_grab_eu_xmltvse  tv_grab_uk_bleb     tv_to_potatoe               yapp
gedi         mimeopen      tv_cat                   tv_grab_fi          tv_grab_uk_tvguide  tv_to_text
[ljohnson@KISE-055 vendor_perl]$ tv_grab_zz_sdjson_sqlite --description
-bash: tv_grab_zz_sdjson_sqlite: command not found
[ljohnson@KISE-055 vendor_perl]$ ./tv_grab_zz_sdjson_sqlite --description
Multinational (Schedules Direct JSON web services with SQLite DB)

Working on a hunch, I added '/usr/bin/vendor_perl' to the path where running 'mythtv-setup' and was able to select "Multinational (Schedules Direct JSON web services with SQLite DB)'

Then I setup the lineup per the instructions naming the source 'Antenna'. Oddly the channel selections were already as desired.

Code: Select all

[ljohnson@KISE-055 .mythtv]$ pwd
/home/ljohnson/.mythtv
[ljohnson@KISE-055 .mythtv]$ ls -lat
total 20756
drwxr-xr-x 10 ljohnson ljohnson     4096 Oct 16 09:07 cache
drwxr-xr-x 10 ljohnson ljohnson     4096 Oct 16 08:55 .
-rw-r--r--  1 ljohnson ljohnson      144 Oct 16 08:55 SD.xmltv
-rw-r--r--  1 ljohnson ljohnson      144 Oct 16 08:51 Antenna.xmltv
drwx------ 26 ljohnson ljohnson     4096 Oct 16 08:41 ..
drwxr-xr-x  4 ljohnson ljohnson     4096 Oct 15 12:09 themes
drwxr-xr-x  4 ljohnson ljohnson     4096 Oct 15 12:09 tmp
-rw-r--r--  1 ljohnson ljohnson 11298459 Oct 15 10:16 tv_grab_zz_sdjson.cache
-rw-r--r--  1 ljohnson ljohnson      124 Oct 15 10:16 tv_grab_zz_sdjson_sqlite.conf
-rw-r--r--  1 ljohnson ljohnson  9883648 Oct 15 10:16 SchedulesDirect.DB
drwxr-xr-x  2 ljohnson ljohnson     4096 May 15 06:28 channels
-rw-r--r--  1 ljohnson ljohnson     2321 May  3 09:50 HD_Antenna.xmltv
-rw-r--r--  1 ljohnson ljohnson     2321 Apr 29 16:26 HD_antenna.xmltv
-rw-r--r--  1 ljohnson ljohnson     2321 Apr 29 16:12 HD_antenna.xmltv.old
drwxr-xr-x  2 root     root         4096 Feb 27  2020 python-scripts
-rw-r--r--  1 ljohnson ljohnson      626 Jun 16  2019 config.xml
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 HardwareProfile
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 MythBrowser
drwxr-xr-x  2 ljohnson ljohnson     4096 Apr 30  2019 3rdParty

HD_Antenna is from a prior source not …sqlite

Will try to finish today, so far, much better ~ thanks

[keepitsimpleengineer]

Ok - got almost every thing working. The SchedulesDirect.DB was being saved in ~/.xmltv vice ~/.mythtv so I fixed that with a symbolic link.

The only problem now is Channel scan only shows 3 of the 8 channels selected ~ yet they all show up easily on the TV when run from the same antenna.
UPDATE: fixed with flag for 'test Decryptability'

Thanks bill6502 & paulh

Hints for others using xmltv from aur.archlinux.org, i.e. archlinux & manjaro &c running mythtv
① grabbers et al will be saved in /usr/bin/vendor_perl ~ I just added /usr/bin/vendor_perl/ to PATH
② watch where tv_grab_zz_sdjson_sqlite & others store results
③ mythfilldatabase requires grabber from ⋯/vendor_perl ~ I just put in a hard link in /usr/bin so updates would fly

Historical note: This mythtv HTPC started out as Ubuntu in `14 but after a number of years I switched to manjaro to get some variety in miseries

[bill6502]

Note that if you only run mythfilldatabase from the command line (as yourself), you're
golden with this setup.

But if the backend runs it automatically, then you'll need an alternative solution. That's
what the alias and ASCII art I linked to above are for. So that both work.

You can point to the location of the SQL DB in the SourceName.xmltv file(s). No need
for links. Putting the DB under ~/.xmltv is a convention, not a requirement.

[keepitsimpleengineer]

Note that if you only run mythfilldatabase from the command line (as yourself), you're
golden with this setup.

But if the backend runs it automatically, then you'll need an alternative solution. That's
what the alias and ASCII art I linked to above are for. So that both work.

You can point to the location of the SQL DB in the SourceName.xmltv file(s). No need
for links. Putting the DB under ~/.xmltv is a convention, not a requirement.