TV Show Metadata
Moderator: Forum Moderators
- jfabernathy
- Senior
- Posts: 577
- Joined: Wed Feb 18, 2015 2:37 pm
- Location: Raleigh, NC
TV Show Metadata
I have on my server a lot of .m4v videos that originally were TV Shows recorded by
mythtv into .ts mpeg2 files and I removed the commercials and converted
them to h.264 and put them on a server using my version of the sendTVtoNAS
script viewtopic.php?f=2&t=1261
Those files are stored on the server in a directory using the program name
and file names that include the program name - title - season and episode
numbers.
i.e. The Big Bang Theory - s1e1 - Pilot.m4v
So anytime I start over with a new mythconverg database and include these
compressed recordings' directory as a Video Storage Group, I get no metadata as I expected, but any new shows I
record, edit, and compress to that same program directory get metadata on
the next day's "mythutil --scanvideos"
The interesting observation is that some of the old videos eventually get
metadata and some of the newly added videos don't.
SO my question is what is supposed to happen with the metadata. And is it
possible to somehow retrieve all the metadata for all the shows that are
NOT in the current mythDB.
mythtv into .ts mpeg2 files and I removed the commercials and converted
them to h.264 and put them on a server using my version of the sendTVtoNAS
script viewtopic.php?f=2&t=1261
Those files are stored on the server in a directory using the program name
and file names that include the program name - title - season and episode
numbers.
i.e. The Big Bang Theory - s1e1 - Pilot.m4v
So anytime I start over with a new mythconverg database and include these
compressed recordings' directory as a Video Storage Group, I get no metadata as I expected, but any new shows I
record, edit, and compress to that same program directory get metadata on
the next day's "mythutil --scanvideos"
The interesting observation is that some of the old videos eventually get
metadata and some of the newly added videos don't.
SO my question is what is supposed to happen with the metadata. And is it
possible to somehow retrieve all the metadata for all the shows that are
NOT in the current mythDB.
Re: TV Show Metadata
As a backup for your metadata so you can always restore it you could create an *.mxml file with the same name as the video file but with the extension replaced by .mxml. That's what I do so when you rescan in Myth it will always prefer the metadata from that file before trying to look it up. That way you just have to set it once and forget about it it will always be restored.
https://www.mythtv.org/wiki/MythTV_Univ ... ata_Format
A simple example for a video called 'Dr. No (1962).mp4' would be a metadata file called 'Dr. No (1962).mxml' which could look something like this (there's some more complete examples on the wiki page above)
https://www.mythtv.org/wiki/MythTV_Univ ... ata_Format
A simple example for a video called 'Dr. No (1962).mp4' would be a metadata file called 'Dr. No (1962).mxml' which could look something like this (there's some more complete examples on the wiki page above)
Code: Select all
<!DOCTYPE MythMetadataXML>
<metadata>
<item>
<title>Dr. No</title>
<description>Sean Connery stars in the movie that began the world famous franchise 50 years ago. Iconic British agent James Bond heads to Jamaica to probe the death of a spy. (1962)(110 mins)
Recorded from Sky 007 HD on Friday 05 October 2012 at 08:00 pm.</description>
<channum>303</channum>
<chansign>Sky 007 HD</chansign>
<startts>Fri, 5 October 2012 20:00:00</startts>
<runtime>120</runtime>
<runtimesecs>7200</runtimesecs>
<categories>
<category name="Movies" type="genre"/>
<category name="Bond Films" type="genre"/>
</categories>
</item>
</metadata>
- dnalorernst
- Developer
- Posts: 105
- Joined: Mon Feb 17, 2020 8:03 pm
Re: TV Show Metadata
Apparently, you have selected this option:
Please look at the frontend logs after doing this scan.
Then, in MythVideo, select a video that does not have good metadata, and press
then
you may get a list if the search gives multiple results, press enter on your choice.
Then wait until all data are fetched, examine the frontend logs and compare them to the logs generated after a "scan for changes".
Again, the wiki is worth to read:
https://www.mythtv.org/wiki/Video_Library
and
https://www.mythtv.org/wiki/MythVideo_File_Parsing
For the second question,
inside MythVideo, press
This fetches a video again, if you previously reset its details.
You may hit by bug [Ticket]#13518[/Ticket]
which I am currently investigating.
Also, you might affected by slow response from the metadata sites:
See last commit from Mark_K, which extends the timeout to 3 minutes.
You can try this patch on your system and report
Anyway, deleting the local cache in
is a good idea.
A note at the end:
It is worth to compare your 'title', 'season', 'episode', 'subtitle' settings with thetvdb.com for validity.
Code: Select all
Setup -> Media Settings -> Video Settings -> Metadata settings -> Perform metadata update after video scan
Then, in MythVideo, select a video that does not have good metadata, and press
Code: Select all
I -> Change Video details -> Reset Details
Code: Select all
I -> Change Video details -> Retrieve Details
Then wait until all data are fetched, examine the frontend logs and compare them to the logs generated after a "scan for changes".
Again, the wiki is worth to read:
https://www.mythtv.org/wiki/Video_Library
and
https://www.mythtv.org/wiki/MythVideo_File_Parsing
For the second question,
inside MythVideo, press
Code: Select all
M -> Retrieve All Details
You may hit by bug [Ticket]#13518[/Ticket]
which I am currently investigating.
Also, you might affected by slow response from the metadata sites:
See last commit from Mark_K, which extends the timeout to 3 minutes.
You can try this patch on your system and report
Anyway, deleting the local cache in
Code: Select all
$HOME/.mythtv/cache/pytmdb3.cache
$HOME/.mythtv/cache/tvdb_api3
A note at the end:
It is worth to compare your 'title', 'season', 'episode', 'subtitle' settings with thetvdb.com for validity.
- jfabernathy
- Senior
- Posts: 577
- Joined: Wed Feb 18, 2015 2:37 pm
- Location: Raleigh, NC
Re: TV Show Metadata
Thanks for all the good suggestions. Here's what I found.
I do have
The frontend log after mythvideo M -> Scan for changes is as below:
This scan only finds shows added to the list that were copied to the Video Storage groups since last scan, which I do daily as a cron job.
The use of does retrieve data for programs that don't have it. The scan time is about 5 seconds per video. However with hundreds of videos that are missing data I'll need an automated way of doing this.
log after:next I tried M -> Retrieve All Details and nothing seems to happen. So I then tried and it updated the one video. Log for both test is:
Maybe the one error at the start is a key?
I do have
Code: Select all
Setup -> Media Settings -> Video Settings -> Metadata settings -> Perform metadata update after video scan
Code: Select all
Sep 27 05:58:20 Marthas-Ubuntu mythfrontend.real: mythfrontend[137431]: I VideoScanner videoscan.cpp:181 (run) Beginning Video Scan.
Sep 27 05:58:20 Marthas-Ubuntu mythfrontend.real: mythfrontend[137431]: I VideoScanner videoscan.cpp:408 (buildFileList) buildFileList directory = myth://Videos@mythbuntu/mnt/md1/samba/public/media/test-vide
os/
Sep 27 05:58:20 Marthas-Ubuntu mythfrontend.real: mythfrontend[137431]: I VideoScanner dirscan.cpp:250 (ScanVideoDirectory) MythVideo::ScanVideoDirectory Scanning Group (myth://Videos@mythbuntu/mnt/md1/samba
/public/media/test-videos/)
Sep 27 05:58:20 Marthas-Ubuntu mythfrontend.real: mythfrontend[137431]: I VideoScanner videoscan.cpp:408 (buildFileList) buildFileList directory = myth://Videos@mythbuntu/mnt/md1/samba/public/media/tv-shows/
Sep 27 05:58:20 Marthas-Ubuntu mythfrontend.real: mythfrontend[137431]: I VideoScanner dirscan.cpp:250 (ScanVideoDirectory) MythVideo::ScanVideoDirectory Scanning Group (myth://Videos@mythbuntu/mnt/md1/samba
/public/media/tv-shows/)
The use of
Code: Select all
I -> Change Video details -> Retrieve Details
log after:
Code: Select all
Sep 27 06:17:26 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I VideoScanner videoscan.cpp:181 (run) Beginning Video Scan.
Sep 27 06:17:26 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I VideoScanner videoscan.cpp:408 (buildFileList) buildFileList directory = myth://Videos@mythbuntu/mnt/md1/samba/public/media/test-videos/
Sep 27 06:17:26 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I VideoScanner dirscan.cpp:250 (ScanVideoDirectory) MythVideo::ScanVideoDirectory Scanning Group (myth://Videos@mythbuntu/mnt/md1/samba/public/media/test-videos/)
Sep 27 06:17:26 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I VideoScanner videoscan.cpp:408 (buildFileList) buildFileList directory = myth://Videos@mythbuntu/mnt/md1/samba/public/media/tv-shows/
Sep 27 06:17:26 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I VideoScanner dirscan.cpp:250 (ScanVideoDirectory) MythVideo::ScanVideoDirectory Scanning Group (myth://Videos@mythbuntu/mnt/md1/samba/public/media/tv-shows/)
Sep 27 06:17:38 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: E CoreContext xmlparsebase.cpp:367 (ParseChildren) XMLParseBase: Parent is NULL
Sep 27 06:17:45 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataDownload metadatagrabber.cpp:424 (RunGrabber) Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N The Big Bang Theory The Fuzzy Boots Corollary
Sep 27 06:17:47 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataDownload metadatacommon.cpp:1141 (ParseMetadataItem) Result Found, Season 1 Episode 3
Sep 27 06:17:48 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataDownload metadatagrabber.cpp:424 (RunGrabber) Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N 80379 The Fuzzy Boots Corollary
Sep 27 06:17:50 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataDownload metadatacommon.cpp:1141 (ParseMetadataItem) Result Found, Season 1 Episode 3
Sep 27 06:17:50 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataDownload metadatadownload.cpp:183 (run) Returning Metadata Results: The Big Bang Theory 1 3
Sep 27 06:17:50 Marthas-Ubuntu mythfrontend.real: mythfrontend[138113]: I MetadataImageDownload metadataimagedownload.cpp:236 (run) Metadata Image Download: http://thetvdb.com/banners/series/80379/episodes/5ed5426665558.jpg -> myth://Screenshots@mythbuntu/The Big Bang Theory Season 1x3_screenshot.jpg
Code: Select all
I -> Change Video details -> Retrieve Details
Code: Select all
Sep 27 06:31:37 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: E CoreContext xmlparsebase.cpp:367 (ParseChildren) XMLParseBase: Parent is NULL
Sep 27 06:31:42 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataDownload metadatagrabber.cpp:424 (RunGrabber) Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N The Big Bang Theory The Luminous Fish Effect
Sep 27 06:31:44 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataDownload metadatacommon.cpp:1141 (ParseMetadataItem) Result Found, Season 1 Episode 4
Sep 27 06:31:44 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataDownload metadatagrabber.cpp:424 (RunGrabber) Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N 80379 The Luminous Fish Effect
Sep 27 06:31:46 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataDownload metadatacommon.cpp:1141 (ParseMetadataItem) Result Found, Season 1 Episode 4
Sep 27 06:31:46 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataDownload metadatadownload.cpp:183 (run) Returning Metadata Results: The Big Bang Theory 1 4
Sep 27 06:31:46 Marthas-Ubuntu mythfrontend.real: mythfrontend[138397]: I MetadataImageDownload metadataimagedownload.cpp:236 (run) Metadata Image Download: http://thetvdb.com/banners/series/80379/episodes/5ed540776e5fb.jpg -> myth://Screenshots@mythbuntu/The Big Bang Theory Season 1x4_screenshot.jpg
- jfabernathy
- Senior
- Posts: 577
- Joined: Wed Feb 18, 2015 2:37 pm
- Location: Raleigh, NC
Re: TV Show Metadata
I've been looking thru log files trying to find something. I did not see anything in either the frontend or backend logs that even acknowledge anything in response to M -> Retrieve All Details
I do find in the frontend log some data when I retrieve the details for an individual video that has no details. I get the details and the log has:I do see one warning in the backend log related to that event:
I do find in the frontend log some data when I retrieve the details for an individual video that has no details. I get the details and the log has:
Code: Select all
2020-09-27 08:10:16.097262 E XMLParseBase: Parent is NULL
2020-09-27 08:10:18.713948 I Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N The Big Bang Theory The Hawking Excitation
2020-09-27 08:10:20.815880 I Result Found, Season 5 Episode 21
2020-09-27 08:10:21.321295 I Running Grabber: /usr/share/mythtv/metadata/Television/ttvdb.py -l en -a US -N 80379 The Hawking Excitation
2020-09-27 08:10:23.069252 I Result Found, Season 5 Episode 21
2020-09-27 08:10:23.069662 I Returning Metadata Results: The Big Bang Theory 5 21
2020-09-27 08:10:23.076236 I Metadata Image Download: http://thetvdb.com/banners/series/80379/episodes/5eb0b2cece018.jpg -> myth://Screenshots@mythbuntu/The Big Bang Theory Season 5x21_screenshot.jpg
Code: Select all
Sep 27 07:57:39 mythbuntu mythbackend: mythbackend[1565]: W ProcessRequest ringbuffer.cpp:664 (Start) RingBuf(/mnt/md0/screenshot//The Big Bang Theory Season 11x18_screenshot.jpg): Not starting read ahead thread, this is a write only RingBuffer
- jfabernathy
- Senior
- Posts: 577
- Joined: Wed Feb 18, 2015 2:37 pm
- Location: Raleigh, NC
Re: TV Show Metadata
I may be missing something, but how do you create the files once all your metadata is created and up to date? I have manually gone through all on one program's episodes and gotten their metadata retrieved. Is there a secret script somewhere that lets you export the metadata from mythtv and create the .mxml files?paulh wrote: ↑Sat Sep 26, 2020 11:27 amAs a backup for your metadata so you can always restore it you could create an *.mxml file with the same name as the video file but with the extension replaced by .mxml. That's what I do so when you rescan in Myth it will always prefer the metadata from that file before trying to look it up. That way you just have to set it once and forget about it it will always be restored.
https://www.mythtv.org/wiki/MythTV_Univ ... ata_Format
A simple example for a video called 'Dr. No (1962).mp4' would be a metadata file called 'Dr. No (1962).mxml' which could look something like this (there's some more complete examples on the wiki page above)
Code: Select all
<!DOCTYPE MythMetadataXML> <metadata> <item> <title>Dr. No</title> <description>Sean Connery stars in the movie that began the world famous franchise 50 years ago. Iconic British agent James Bond heads to Jamaica to probe the death of a spy. (1962)(110 mins) Recorded from Sky 007 HD on Friday 05 October 2012 at 08:00 pm.</description> <channum>303</channum> <chansign>Sky 007 HD</chansign> <startts>Fri, 5 October 2012 20:00:00</startts> <runtime>120</runtime> <runtimesecs>7200</runtimesecs> <categories> <category name="Movies" type="genre"/> <category name="Bond Films" type="genre"/> </categories> </item> </metadata>
- dnalorernst
- Developer
- Posts: 105
- Joined: Mon Feb 17, 2020 8:03 pm
Re: TV Show Metadata
Please try and check the attached script fits to your needs.
Roland
Roland
- Attachments
-
- export_to_mxml.zip
- Initial version to export metadata into mxml files
- (2.38 KiB) Downloaded 71 times
- jfabernathy
- Senior
- Posts: 577
- Joined: Wed Feb 18, 2015 2:37 pm
- Location: Raleigh, NC
Re: TV Show Metadata
Thanks, I'll play with it.