[Solved] Some channels showing as "NO DATA" on MythWeb
Moderator: Forum Moderators
-
- Junior
- Posts: 18
- Joined: Sat Dec 26, 2020 5:58 pm
[Solved] Some channels showing as "NO DATA" on MythWeb
I have a problem with my listings on MythWeb, which is that several channels are showing as "NO DATA".
I definitely do have data. I have checked in the MySQL tables, and also if I look at the guide on either a Kodi frontend or a MythTV frontend, then I can see program data for all channels. The problem seems to be specific to MythWeb.
A little bit of googling has shown that I'm not the first person to have this problem, though I'm not sure I have enough information to solve it.
This thread has the same symptoms that I have, but that seems to be in the context of XMLTV. I use EIT data for my listings, so I don't think it can be the same problem.
viewtopic.php?t=3812
This one maybe looks a bit closer:
https://lists.archive.carbon60.com/mythtv/users/129335
It might not be unfair to say that my channel table is messed up. I've been using MythTV for about 12 years, and have kept the same backend database all that time (with appropriate upgrades where necessary), and I've noticed that many channels have several entries for the same channel, all but one of which are marked as deleted. If my understanding is correct, MythTV doesn't actually delete channels from the table when you "delete" them from the channel editor, it just marks them as deleted. I've done so many retunings over the course of those 12 years it's not surprising the channel table is a bit of a mess.
Is there a way I can de-messify my channels table? Or might that not be the problem anyway?
I'm running MythWeb 31.0 (and indeed the rest of MythBackend 31.0) on Ubuntu 20.04.
Many thanks for any suggestions
Adam
I definitely do have data. I have checked in the MySQL tables, and also if I look at the guide on either a Kodi frontend or a MythTV frontend, then I can see program data for all channels. The problem seems to be specific to MythWeb.
A little bit of googling has shown that I'm not the first person to have this problem, though I'm not sure I have enough information to solve it.
This thread has the same symptoms that I have, but that seems to be in the context of XMLTV. I use EIT data for my listings, so I don't think it can be the same problem.
viewtopic.php?t=3812
This one maybe looks a bit closer:
https://lists.archive.carbon60.com/mythtv/users/129335
It might not be unfair to say that my channel table is messed up. I've been using MythTV for about 12 years, and have kept the same backend database all that time (with appropriate upgrades where necessary), and I've noticed that many channels have several entries for the same channel, all but one of which are marked as deleted. If my understanding is correct, MythTV doesn't actually delete channels from the table when you "delete" them from the channel editor, it just marks them as deleted. I've done so many retunings over the course of those 12 years it's not surprising the channel table is a bit of a mess.
Is there a way I can de-messify my channels table? Or might that not be the problem anyway?
I'm running MythWeb 31.0 (and indeed the rest of MythBackend 31.0) on Ubuntu 20.04.
Many thanks for any suggestions
Adam
Last edited by AdamJacobs on Sat Jan 08, 2022 5:52 pm, edited 1 time in total.
Re: Some channels showing as "NO DATA" on MythWeb
Hi Adam,
I too have the same problem and I agree that it is related to channel setup.
I have a UK setup using DVB-T and EIT off-air scheduling data from Crystal Palace (London) transmitter.
With my 'Live system' - an AMD box with Hauppauge PCI /USB tuners, Ubuntu 14.04 and 0.27, Mythweb works fine.
My replacement 'test' system is an Intel 'NUC' combined frontend/backend running xubuntu 20.04 and myth 31 with HD Homerun quad DVB-T/T2 tuners in the loft. Mythweb shows the same symptoms as you with 'no data' on many channels.
Your comments about your database struck a chord: my database, like yours is also 'mature' - dating back to 2009 - with spurious channels in it.
When I imported a load of recordings and the database from the 'live' system I cleared all channel information and retuned with the Homerun tuners. In the retune the channelid's changed: BBC Two for example was 1002 but became 20002. That meant that my old recordings were unhelpfully tagged with a number like (I recall) #1002#1002 rather that 'BBC Two'.
My fix was to re-import the database and retain the old channels as well as tuning the new ones. I then deleted the 'old' junk channels like shopping and 'Adult' ones with no recordings.
Today I tried Mythweb for the first time with the test system to find that it is broken as you describe. Popular channels have no scheduling data, the junk ones do. My channel editor https://www.mythtv.org/wiki/Channel_Editor showed an interesting pattern - only the channels with both old and new entries fail with Mythweb.
To nail this completely, I took a sample channel with two entries - Callsign=Smithsonian, channel number=56, channel id's of 1701 (old Hauppauge) and 20056 (new Homeruns) both with a serviceid of 23152. If I hide the old Hauppauge channel entry, reboot and run Mythweb then the Smithsonian data magically appears!!
So the choice appears to be losing channel names on old recordings or having no scheduling data in Mythweb. Unless of course there is some way of populating the channel id's with the callsign (or Channel Names - they are the same in the UK but not elsewhere).
Does this match your findings?
Phil
I too have the same problem and I agree that it is related to channel setup.
I have a UK setup using DVB-T and EIT off-air scheduling data from Crystal Palace (London) transmitter.
With my 'Live system' - an AMD box with Hauppauge PCI /USB tuners, Ubuntu 14.04 and 0.27, Mythweb works fine.
My replacement 'test' system is an Intel 'NUC' combined frontend/backend running xubuntu 20.04 and myth 31 with HD Homerun quad DVB-T/T2 tuners in the loft. Mythweb shows the same symptoms as you with 'no data' on many channels.
Your comments about your database struck a chord: my database, like yours is also 'mature' - dating back to 2009 - with spurious channels in it.
When I imported a load of recordings and the database from the 'live' system I cleared all channel information and retuned with the Homerun tuners. In the retune the channelid's changed: BBC Two for example was 1002 but became 20002. That meant that my old recordings were unhelpfully tagged with a number like (I recall) #1002#1002 rather that 'BBC Two'.
My fix was to re-import the database and retain the old channels as well as tuning the new ones. I then deleted the 'old' junk channels like shopping and 'Adult' ones with no recordings.
Today I tried Mythweb for the first time with the test system to find that it is broken as you describe. Popular channels have no scheduling data, the junk ones do. My channel editor https://www.mythtv.org/wiki/Channel_Editor showed an interesting pattern - only the channels with both old and new entries fail with Mythweb.
To nail this completely, I took a sample channel with two entries - Callsign=Smithsonian, channel number=56, channel id's of 1701 (old Hauppauge) and 20056 (new Homeruns) both with a serviceid of 23152. If I hide the old Hauppauge channel entry, reboot and run Mythweb then the Smithsonian data magically appears!!
So the choice appears to be losing channel names on old recordings or having no scheduling data in Mythweb. Unless of course there is some way of populating the channel id's with the callsign (or Channel Names - they are the same in the UK but not elsewhere).
Does this match your findings?
Phil
Re: Some channels showing as "NO DATA" on MythWeb
Maybe this helps in understanding.
(1) You never need to delete old channels manually. They will be deleted automatically when there there are no more recordings referring to them. This means that you will still see the channel name instead of #1002#1002 (or so) even when the channel is now on a different location.
(2) Install the very latest version of MythWeb. Old versions also show the channels marked as deleted. I think that are the channels that show up as having "no data" which is correct because there is no new guide information for deleted channels. This was fixed in Mythweb a few months ago. As of last week or so all known bugs have been fixed so I really advise to install the latest.
Hope this helps,
Klaas.
(1) You never need to delete old channels manually. They will be deleted automatically when there there are no more recordings referring to them. This means that you will still see the channel name instead of #1002#1002 (or so) even when the channel is now on a different location.
(2) Install the very latest version of MythWeb. Old versions also show the channels marked as deleted. I think that are the channels that show up as having "no data" which is correct because there is no new guide information for deleted channels. This was fixed in Mythweb a few months ago. As of last week or so all known bugs have been fixed so I really advise to install the latest.
Hope this helps,
Klaas.
Re: Some channels showing as "NO DATA" on MythWeb
Hi Klaas,
thanks for the reply. Sorry for the delay in replying but I have been pondering!
Mythweb was turned on via control panel.
All dates in ls -l /var/www/html/mythweb are either 8 Nov or 24 Dec 2021.
sudo apt-get update and sudo apt-get upgrade mythweb shows:
"mythweb is already the newest version (2:31.0+fixes.202111081900.25f1bb1d12~ubuntu20.04.1)".
My setup has two video sources.
Sourceid=1 called EIT. Originally set up for the old Hauppauge tuners. No capture devices.
I have left this source and its channels in place in order to retain channel names/callsigns for old recordings.
Sourceid=2 called HDHR - set up and working with the 4 HD Homerun tuners.
Any channels in source1 which are NOT hidden or deleted mask the source2 listings in Mythweb.
Two questions:
SourceId=1 was set for EIT grabbing - I have just set it to 'no listings grabber' but could this have been the problem?
Would removing source1 completely resolve the mythweb issue without losing the channel names from old recordings?
Thanks again.
Phil
thanks for the reply. Sorry for the delay in replying but I have been pondering!
That's very interesting but it is a little more complex? When is that rule triggered? What prevents it deleting all channels on a new installation before any recordings have been made? There are some channels I have never ever recorded from (eg shopping channels) - why are they still there?You never need to delete old channels manually. They will be deleted automatically when there there are no more recordings referring to them
I think it is up to date. I loaded xubuntu on this system on 30 Sept 2021 and Mythtv 31 subsequently.Install the very latest version of MythWeb
Mythweb was turned on via control panel.
All dates in ls -l /var/www/html/mythweb are either 8 Nov or 24 Dec 2021.
sudo apt-get update and sudo apt-get upgrade mythweb shows:
"mythweb is already the newest version (2:31.0+fixes.202111081900.25f1bb1d12~ubuntu20.04.1)".
My setup has two video sources.
Sourceid=1 called EIT. Originally set up for the old Hauppauge tuners. No capture devices.
I have left this source and its channels in place in order to retain channel names/callsigns for old recordings.
Sourceid=2 called HDHR - set up and working with the 4 HD Homerun tuners.
Any channels in source1 which are NOT hidden or deleted mask the source2 listings in Mythweb.
Two questions:
SourceId=1 was set for EIT grabbing - I have just set it to 'no listings grabber' but could this have been the problem?
Would removing source1 completely resolve the mythweb issue without losing the channel names from old recordings?
Thanks again.
Phil
Re: Some channels showing as "NO DATA" on MythWeb
About deleting channels. This deserves a better explanation.
What I meant was that it is never needed to remove channels from the mythconverg database using mysql commands.
Channels that are deleted with mythtv-setup are not immediately removed from the database but are marked as deleted in the database.
When there are no recordings anymore from these channels then these channels will be removed from the database by mythbackend.
Again, this is only for channels that have been deleted earlier with mythtv-setup.
Channels that are NOT deleted will not be removed from the database, even if they do not ever have made recordings.
Hope this helps!
Klaas.
What I meant was that it is never needed to remove channels from the mythconverg database using mysql commands.
Channels that are deleted with mythtv-setup are not immediately removed from the database but are marked as deleted in the database.
When there are no recordings anymore from these channels then these channels will be removed from the database by mythbackend.
Again, this is only for channels that have been deleted earlier with mythtv-setup.
Channels that are NOT deleted will not be removed from the database, even if they do not ever have made recordings.
Yes, I think this is the cause of your problems. Because the channels of this video source are still present you now have each channel twice, one without guide data and one with guide data. You can, with mythtv-setup, delete all channels of the EIT video source or delete the EIT video source which will also delete all channels (which means flag them in the database as deleted) and these channels will only be removed from the database when there are no more recordings from these channels.My setup has two video sources.
Sourceid=1 called EIT. Originally set up for the old Hauppauge tuners. No capture devices.
I have left this source and its channels in place in order to retain channel names/callsigns for old recordings.
Sourceid=2 called HDHR - set up and working with the 4 HD Homerun tuners.
Any channels in source1 which are NOT hidden or deleted mask the source2 listings in Mythweb.
Two questions:
SourceId=1 was set for EIT grabbing - I have just set it to 'no listings grabber' but could this have been the problem?
Would removing source1 completely resolve the mythweb issue without losing the channel names from old recordings?
Hope this helps!
Klaas.
Re: Some channels showing as "NO DATA" on MythWeb
Ah!! Thanks for your very patient explanations!!
It is now fixed. It was sufficient to merely 'hide' the channels in the old source and then it all works. Very old recordings still have the name of the channel and yet Mythweb now shows listings correctly for source 2.
I was a little cautious about deleting the channels or the old source - I'll need to re-import recordings and database from the old live system at some stage in the near future and will try source deletion to satisfy my curiosity first!
Many thanks again!
Adam: I tried just hiding BBC1 first (because that step is reversible) to check then when that worked hid all channels in source1. The Perl based channel editor https://www.mythtv.org/wiki/Channel_Editor allowed me to see the situation. A quick way of hiding in bulk is: sort by sourceid and identify the offending one; note the line numbers (in my case 0 to 102 inclusive) then edit > bulk edit> 0:102, Visible, Clear. Then file > update database.
Sorry if this appears to be an advert! Shout if you need clarification.
Phil
It is now fixed. It was sufficient to merely 'hide' the channels in the old source and then it all works. Very old recordings still have the name of the channel and yet Mythweb now shows listings correctly for source 2.
I was a little cautious about deleting the channels or the old source - I'll need to re-import recordings and database from the old live system at some stage in the near future and will try source deletion to satisfy my curiosity first!
Many thanks again!
Adam: I tried just hiding BBC1 first (because that step is reversible) to check then when that worked hid all channels in source1. The Perl based channel editor https://www.mythtv.org/wiki/Channel_Editor allowed me to see the situation. A quick way of hiding in bulk is: sort by sourceid and identify the offending one; note the line numbers (in my case 0 to 102 inclusive) then edit > bulk edit> 0:102, Visible, Clear. Then file > update database.
Sorry if this appears to be an advert! Shout if you need clarification.
Phil
Re: Some channels showing as "NO DATA" on MythWeb
I have investigated further with perl code to list recordings and have done database backups and restores to establish this (at least to my satisfaction):-
In order to ensure that old recordings retain their original channel names AND Mythweb shows scheduling data then you have two options:
1. Delete the 'old' source (backend setup > sources > select old source > D). That is the source without capture cards.
---- OR ---
2. retain the old source but hide all the channels in it.
However,
---- DO NOT --- delete channels in the old source in either case.
hth
Phil
In order to ensure that old recordings retain their original channel names AND Mythweb shows scheduling data then you have two options:
1. Delete the 'old' source (backend setup > sources > select old source > D). That is the source without capture cards.
---- OR ---
2. retain the old source but hide all the channels in it.
However,
---- DO NOT --- delete channels in the old source in either case.
hth
Phil
-
- Junior
- Posts: 18
- Joined: Sat Dec 26, 2020 5:58 pm
Re: Some channels showing as "NO DATA" on MythWeb
Thanks both, but I'm not sure this gets me any further forward.
I do have old channels associated with an old source, but I have already deleted the old source. I only have 1 source active.
And as far as I can tell, I already have the latest version of mythweb. Typing sudo apt upgrade mythweb gives me this:
mythweb is already the newest version (2:31.0+fixes.20200323.9579662cdc-0ubuntu1)
Unless I need to get it from a repository that has a newer version than the standard Ubuntu 20.04 one?
I do have old channels associated with an old source, but I have already deleted the old source. I only have 1 source active.
And as far as I can tell, I already have the latest version of mythweb. Typing sudo apt upgrade mythweb gives me this:
mythweb is already the newest version (2:31.0+fixes.20200323.9579662cdc-0ubuntu1)
Unless I need to get it from a repository that has a newer version than the standard Ubuntu 20.04 one?
Re: Some channels showing as "NO DATA" on MythWeb
That is the version of MythWeb from the official Ubuntu repository which was current when 20.04 was released but is missing all the updates since then.
To get a more up to date version you can use the MythBuntu PPA which is rebuilt whenever the 31-fixes version gets updated (it's actually fairly rare it gets updated but when it does the fixes are often important ones that get backported from the current development branch).
All you need to do to add the PPA is run this command
then do the usual system upgrade to install any updates
To get a more up to date version you can use the MythBuntu PPA which is rebuilt whenever the 31-fixes version gets updated (it's actually fairly rare it gets updated but when it does the fixes are often important ones that get backported from the current development branch).
All you need to do to add the PPA is run this command
Code: Select all
sudo add-apt-repository ppa:mythbuntu/31
Code: Select all
sudo apt upgrade
Re: Some channels showing as "NO DATA" on MythWeb
Thanks Klaas and Paul,
I am seeing two issues here, both associated with importing recordings and a database from an old 'live' system to a development one;
The old system has Hauppauge PCI/USB tuners, the development one has networked HD Homerun quad tuners.
1. When 'old' channels are deleted the recordings previously made on those channels lose their channel name (or Callsign - they are same here in UK). Klaas has said:
instead of 'BBC Two'.
If that bug did not exist then I would not have seen bug 2:
2. I have just done that update again - sudo apt-upgrade mythweb shows it was fully updated:
I can flip it on and off.
Similarly, if I delete an old channel then the scheduling data is shown.
Oddly, frontend shows listing data correctly in all circumstances.
So, to summarise:
Delete old channels: channel names not shown in old recordings, but Mythweb shows new channel listings.
Retain old channels and have them visible: Recordings show channel name but Mythweb shows 'old' expired 'No data' listing data.
Retain old channels but hide them; recordings correctly show channel name; 'new' listing data shows on Mythweb.
Retain old channels but delete the old source: ditto.
I am on a development system which I can adjust without upsetting anyone. Can I provide further information to help pin this down?
Phil
I am seeing two issues here, both associated with importing recordings and a database from an old 'live' system to a development one;
The old system has Hauppauge PCI/USB tuners, the development one has networked HD Homerun quad tuners.
1. When 'old' channels are deleted the recordings previously made on those channels lose their channel name (or Callsign - they are same here in UK). Klaas has said:
That would be very sensible and very desirable behaviour. It means that when changing tuning data in a big way then everything still works. Unfortunately it is not what I am seeing with myth 31.20200101-1. If I delete a channel which no longer has a capture device, either with backend setup or via the api interface, then previous recordings from that channel no longer show the channel name - I get a string like #1002#1002You never need to delete old channels manually. They will be deleted automatically when there are no
more recordings referring to them. This means that you will still see the channel name instead of #1002#1002
(or so) even when the channel is now on a different location.
instead of 'BBC Two'.
If that bug did not exist then I would not have seen bug 2:
2. I have just done that update again - sudo apt-upgrade mythweb shows it was fully updated:
Unfortunately, the scheduling data shown in Mythweb is still broken. If the 'old' channels are visible then the scheduling data is missing (ie it is showing the old Hauppauge tuner info which is now out of date). If I hide all channels in the old source then the new HD Homerun source data is correctly shown.mythweb is already the newest version (2:31.0+fixes.202111081900.25f1bb1d12~ubuntu20.04.1)
I can flip it on and off.
Similarly, if I delete an old channel then the scheduling data is shown.
Oddly, frontend shows listing data correctly in all circumstances.
So, to summarise:
Delete old channels: channel names not shown in old recordings, but Mythweb shows new channel listings.
Retain old channels and have them visible: Recordings show channel name but Mythweb shows 'old' expired 'No data' listing data.
Retain old channels but hide them; recordings correctly show channel name; 'new' listing data shows on Mythweb.
Retain old channels but delete the old source: ditto.
I am on a development system which I can adjust without upsetting anyone. Can I provide further information to help pin this down?
Phil
Re: Some channels showing as "NO DATA" on MythWeb
This looks like it is rather old.It should have a similar date to that of mythweb, so from late 2021.Unfortunately it is not what I am seeing with myth 31.20200101-1.
Can you please update this also to the latest and try again deleting a channel with the mythtv-setup Channel Editor?
Recordings from that channel should then have, in mythfrontend, still the channel name and not the #1002#1002 or so.
Klaas.
Re: Some channels showing as "NO DATA" on MythWeb
Sorry - correction. The API interface is showing that version.
Frontend > info shows that it is
fixes/31+fixes.202111081900.25f1bb1d12~ubuntu20.04.1
Phil
Frontend > info shows that it is
fixes/31+fixes.202111081900.25f1bb1d12~ubuntu20.04.1
Phil
Re: Some channels showing as "NO DATA" on MythWeb
Ouch! lots of egg on my face now.
With version 31.20200101-1 deleting channels definitely caused the channel name to be lost from recordings.
With 31.201108 it is fixed - proved with:
1. make all channels visible.
2. Delete channel 4 from both sources - old recordings still show channel name.
3. Delete channel 5 with perl based editor -old recordings still show channel name
4. restored the 3am database backup.
The recordings with channels deleted under 31.20200101-1 have not recovered their channel names.
So the advice to the OP is: Upgrade to latest Mythtv. Delete old redundant channels which no longer have scheduling data.
Thanks Klaas and Paul for your patience!
Phil
With version 31.20200101-1 deleting channels definitely caused the channel name to be lost from recordings.
With 31.201108 it is fixed - proved with:
1. make all channels visible.
2. Delete channel 4 from both sources - old recordings still show channel name.
3. Delete channel 5 with perl based editor -old recordings still show channel name
4. restored the 3am database backup.
The recordings with channels deleted under 31.20200101-1 have not recovered their channel names.
So the advice to the OP is: Upgrade to latest Mythtv. Delete old redundant channels which no longer have scheduling data.
Thanks Klaas and Paul for your patience!
Phil
Re: Some channels showing as "NO DATA" on MythWeb
I have updated the wiki: https://www.mythtv.org/wiki/MythWeb section 6.17.
Would someone proof-read it please?
Thanks
Phil
Would someone proof-read it please?
Thanks
Phil
Re: Some channels showing as "NO DATA" on MythWeb
Hi Phil,
Good that you do write it down in the Wiki. Appreciated!
You might want to add that also MythWeb needs to be a very recent version. As it is a different package it is not automatically updated when you update mythtv itself.
Note that in mythtv-setup there is a button for deleting all channels of a video source. See https://www.mythtv.org/wiki/Channel_Scanning#Delete
Thanks,
Klaas.
Good that you do write it down in the Wiki. Appreciated!
You might want to add that also MythWeb needs to be a very recent version. As it is a different package it is not automatically updated when you update mythtv itself.
Note that in mythtv-setup there is a button for deleting all channels of a video source. See https://www.mythtv.org/wiki/Channel_Scanning#Delete
Thanks,
Klaas.