[Solved] Some channels showing as "NO DATA" on MythWeb

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

AdamJacobs
Junior
Posts: 18
Joined: Sat Dec 26, 2020 5:58 pm
Great Britain

[Solved] Some channels showing as "NO DATA" on MythWeb

Post by AdamJacobs »

I have a problem with my listings on MythWeb, which is that several channels are showing as "NO DATA".

I definitely do have data. I have checked in the MySQL tables, and also if I look at the guide on either a Kodi frontend or a MythTV frontend, then I can see program data for all channels. The problem seems to be specific to MythWeb.

A little bit of googling has shown that I'm not the first person to have this problem, though I'm not sure I have enough information to solve it.

This thread has the same symptoms that I have, but that seems to be in the context of XMLTV. I use EIT data for my listings, so I don't think it can be the same problem.

viewtopic.php?t=3812

This one maybe looks a bit closer:

https://lists.archive.carbon60.com/mythtv/users/129335

It might not be unfair to say that my channel table is messed up. I've been using MythTV for about 12 years, and have kept the same backend database all that time (with appropriate upgrades where necessary), and I've noticed that many channels have several entries for the same channel, all but one of which are marked as deleted. If my understanding is correct, MythTV doesn't actually delete channels from the table when you "delete" them from the channel editor, it just marks them as deleted. I've done so many retunings over the course of those 12 years it's not surprising the channel table is a bit of a mess.

Is there a way I can de-messify my channels table? Or might that not be the problem anyway?

I'm running MythWeb 31.0 (and indeed the rest of MythBackend 31.0) on Ubuntu 20.04.

Many thanks for any suggestions

Adam
Last edited by AdamJacobs on Sat Jan 08, 2022 5:52 pm, edited 1 time in total.
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Hi Adam,
I too have the same problem and I agree that it is related to channel setup.
I have a UK setup using DVB-T and EIT off-air scheduling data from Crystal Palace (London) transmitter.

With my 'Live system' - an AMD box with Hauppauge PCI /USB tuners, Ubuntu 14.04 and 0.27, Mythweb works fine.

My replacement 'test' system is an Intel 'NUC' combined frontend/backend running xubuntu 20.04 and myth 31 with HD Homerun quad DVB-T/T2 tuners in the loft. Mythweb shows the same symptoms as you with 'no data' on many channels.

Your comments about your database struck a chord: my database, like yours is also 'mature' - dating back to 2009 - with spurious channels in it.

When I imported a load of recordings and the database from the 'live' system I cleared all channel information and retuned with the Homerun tuners. In the retune the channelid's changed: BBC Two for example was 1002 but became 20002. That meant that my old recordings were unhelpfully tagged with a number like (I recall) #1002#1002 rather that 'BBC Two'.

My fix was to re-import the database and retain the old channels as well as tuning the new ones. I then deleted the 'old' junk channels like shopping and 'Adult' ones with no recordings.

Today I tried Mythweb for the first time with the test system to find that it is broken as you describe. Popular channels have no scheduling data, the junk ones do. My channel editor https://www.mythtv.org/wiki/Channel_Editor showed an interesting pattern - only the channels with both old and new entries fail with Mythweb.

To nail this completely, I took a sample channel with two entries - Callsign=Smithsonian, channel number=56, channel id's of 1701 (old Hauppauge) and 20056 (new Homeruns) both with a serviceid of 23152. If I hide the old Hauppauge channel entry, reboot and run Mythweb then the Smithsonian data magically appears!!

So the choice appears to be losing channel names on old recordings or having no scheduling data in Mythweb. Unless of course there is some way of populating the channel id's with the callsign (or Channel Names - they are the same in the UK but not elsewhere).

Does this match your findings?

Phil
User avatar
kmdewaal
Developer
Posts: 640
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: Some channels showing as "NO DATA" on MythWeb

Post by kmdewaal »

Maybe this helps in understanding.
(1) You never need to delete old channels manually. They will be deleted automatically when there there are no more recordings referring to them. This means that you will still see the channel name instead of #1002#1002 (or so) even when the channel is now on a different location.
(2) Install the very latest version of MythWeb. Old versions also show the channels marked as deleted. I think that are the channels that show up as having "no data" which is correct because there is no new guide information for deleted channels. This was fixed in Mythweb a few months ago. As of last week or so all known bugs have been fixed so I really advise to install the latest.

Hope this helps,
Klaas.
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Hi Klaas,
thanks for the reply. Sorry for the delay in replying but I have been pondering!
You never need to delete old channels manually. They will be deleted automatically when there there are no more recordings referring to them
That's very interesting but it is a little more complex? When is that rule triggered? What prevents it deleting all channels on a new installation before any recordings have been made? There are some channels I have never ever recorded from (eg shopping channels) - why are they still there?
Install the very latest version of MythWeb
I think it is up to date. I loaded xubuntu on this system on 30 Sept 2021 and Mythtv 31 subsequently.
Mythweb was turned on via control panel.
All dates in ls -l /var/www/html/mythweb are either 8 Nov or 24 Dec 2021.
sudo apt-get update and sudo apt-get upgrade mythweb shows:
"mythweb is already the newest version (2:31.0+fixes.202111081900.25f1bb1d12~ubuntu20.04.1)".

My setup has two video sources.
Sourceid=1 called EIT. Originally set up for the old Hauppauge tuners. No capture devices.
I have left this source and its channels in place in order to retain channel names/callsigns for old recordings.

Sourceid=2 called HDHR - set up and working with the 4 HD Homerun tuners.

Any channels in source1 which are NOT hidden or deleted mask the source2 listings in Mythweb.

Two questions:
SourceId=1 was set for EIT grabbing - I have just set it to 'no listings grabber' but could this have been the problem?
Would removing source1 completely resolve the mythweb issue without losing the channel names from old recordings?

Thanks again.
Phil
User avatar
kmdewaal
Developer
Posts: 640
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: Some channels showing as "NO DATA" on MythWeb

Post by kmdewaal »

About deleting channels. This deserves a better explanation.
What I meant was that it is never needed to remove channels from the mythconverg database using mysql commands.
Channels that are deleted with mythtv-setup are not immediately removed from the database but are marked as deleted in the database.
When there are no recordings anymore from these channels then these channels will be removed from the database by mythbackend.
Again, this is only for channels that have been deleted earlier with mythtv-setup.
Channels that are NOT deleted will not be removed from the database, even if they do not ever have made recordings.
My setup has two video sources.
Sourceid=1 called EIT. Originally set up for the old Hauppauge tuners. No capture devices.
I have left this source and its channels in place in order to retain channel names/callsigns for old recordings.

Sourceid=2 called HDHR - set up and working with the 4 HD Homerun tuners.

Any channels in source1 which are NOT hidden or deleted mask the source2 listings in Mythweb.

Two questions:
SourceId=1 was set for EIT grabbing - I have just set it to 'no listings grabber' but could this have been the problem?
Would removing source1 completely resolve the mythweb issue without losing the channel names from old recordings?
Yes, I think this is the cause of your problems. Because the channels of this video source are still present you now have each channel twice, one without guide data and one with guide data. You can, with mythtv-setup, delete all channels of the EIT video source or delete the EIT video source which will also delete all channels (which means flag them in the database as deleted) and these channels will only be removed from the database when there are no more recordings from these channels.

Hope this helps!
Klaas.
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Ah!! Thanks for your very patient explanations!!
It is now fixed. It was sufficient to merely 'hide' the channels in the old source and then it all works. Very old recordings still have the name of the channel and yet Mythweb now shows listings correctly for source 2.
I was a little cautious about deleting the channels or the old source - I'll need to re-import recordings and database from the old live system at some stage in the near future and will try source deletion to satisfy my curiosity first!
Many thanks again!

Adam: I tried just hiding BBC1 first (because that step is reversible) to check then when that worked hid all channels in source1. The Perl based channel editor https://www.mythtv.org/wiki/Channel_Editor allowed me to see the situation. A quick way of hiding in bulk is: sort by sourceid and identify the offending one; note the line numbers (in my case 0 to 102 inclusive) then edit > bulk edit> 0:102, Visible, Clear. Then file > update database.
Sorry if this appears to be an advert! Shout if you need clarification.
Phil
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

I have investigated further with perl code to list recordings and have done database backups and restores to establish this (at least to my satisfaction):-

In order to ensure that old recordings retain their original channel names AND Mythweb shows scheduling data then you have two options:

1. Delete the 'old' source (backend setup > sources > select old source > D). That is the source without capture cards.
---- OR ---
2. retain the old source but hide all the channels in it.

However,
---- DO NOT --- delete channels in the old source in either case.

hth
Phil
AdamJacobs
Junior
Posts: 18
Joined: Sat Dec 26, 2020 5:58 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by AdamJacobs »

Thanks both, but I'm not sure this gets me any further forward.

I do have old channels associated with an old source, but I have already deleted the old source. I only have 1 source active.

And as far as I can tell, I already have the latest version of mythweb. Typing sudo apt upgrade mythweb gives me this:

mythweb is already the newest version (2:31.0+fixes.20200323.9579662cdc-0ubuntu1)

Unless I need to get it from a repository that has a newer version than the standard Ubuntu 20.04 one?
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by paulh »

That is the version of MythWeb from the official Ubuntu repository which was current when 20.04 was released but is missing all the updates since then.

To get a more up to date version you can use the MythBuntu PPA which is rebuilt whenever the 31-fixes version gets updated (it's actually fairly rare it gets updated but when it does the fixes are often important ones that get backported from the current development branch).

All you need to do to add the PPA is run this command

Code: Select all

sudo add-apt-repository ppa:mythbuntu/31
then do the usual system upgrade to install any updates

Code: Select all

sudo apt upgrade
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Thanks Klaas and Paul,

I am seeing two issues here, both associated with importing recordings and a database from an old 'live' system to a development one;
The old system has Hauppauge PCI/USB tuners, the development one has networked HD Homerun quad tuners.

1. When 'old' channels are deleted the recordings previously made on those channels lose their channel name (or Callsign - they are same here in UK). Klaas has said:
You never need to delete old channels manually. They will be deleted automatically when there are no
more recordings referring to them. This means that you will still see the channel name instead of #1002#1002
(or so) even when the channel is now on a different location.
That would be very sensible and very desirable behaviour. It means that when changing tuning data in a big way then everything still works. Unfortunately it is not what I am seeing with myth 31.20200101-1. If I delete a channel which no longer has a capture device, either with backend setup or via the api interface, then previous recordings from that channel no longer show the channel name - I get a string like #1002#1002
instead of 'BBC Two'.
If that bug did not exist then I would not have seen bug 2:

2. I have just done that update again - sudo apt-upgrade mythweb shows it was fully updated:
mythweb is already the newest version (2:31.0+fixes.202111081900.25f1bb1d12~ubuntu20.04.1)
Unfortunately, the scheduling data shown in Mythweb is still broken. If the 'old' channels are visible then the scheduling data is missing (ie it is showing the old Hauppauge tuner info which is now out of date). If I hide all channels in the old source then the new HD Homerun source data is correctly shown.
I can flip it on and off.
Similarly, if I delete an old channel then the scheduling data is shown.
Oddly, frontend shows listing data correctly in all circumstances.

So, to summarise:
Delete old channels: channel names not shown in old recordings, but Mythweb shows new channel listings.
Retain old channels and have them visible: Recordings show channel name but Mythweb shows 'old' expired 'No data' listing data.
Retain old channels but hide them; recordings correctly show channel name; 'new' listing data shows on Mythweb.
Retain old channels but delete the old source: ditto.

I am on a development system which I can adjust without upsetting anyone. Can I provide further information to help pin this down?

Phil
User avatar
kmdewaal
Developer
Posts: 640
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: Some channels showing as "NO DATA" on MythWeb

Post by kmdewaal »

Unfortunately it is not what I am seeing with myth 31.20200101-1.
This looks like it is rather old.It should have a similar date to that of mythweb, so from late 2021.
Can you please update this also to the latest and try again deleting a channel with the mythtv-setup Channel Editor?
Recordings from that channel should then have, in mythfrontend, still the channel name and not the #1002#1002 or so.

Klaas.
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Sorry - correction. The API interface is showing that version.
Frontend > info shows that it is
fixes/31+fixes.202111081900.25f1bb1d12~ubuntu20.04.1
Phil
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

Ouch! lots of egg on my face now.
With version 31.20200101-1 deleting channels definitely caused the channel name to be lost from recordings.
With 31.201108 it is fixed - proved with:
1. make all channels visible.
2. Delete channel 4 from both sources - old recordings still show channel name.
3. Delete channel 5 with perl based editor -old recordings still show channel name
4. restored the 3am database backup.
The recordings with channels deleted under 31.20200101-1 have not recovered their channel names.

So the advice to the OP is: Upgrade to latest Mythtv. Delete old redundant channels which no longer have scheduling data.

Thanks Klaas and Paul for your patience!
Phil
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: Some channels showing as "NO DATA" on MythWeb

Post by PhilB »

I have updated the wiki: https://www.mythtv.org/wiki/MythWeb section 6.17.
Would someone proof-read it please?
Thanks
Phil
User avatar
kmdewaal
Developer
Posts: 640
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: Some channels showing as "NO DATA" on MythWeb

Post by kmdewaal »

Hi Phil,

Good that you do write it down in the Wiki. Appreciated!
You might want to add that also MythWeb needs to be a very recent version. As it is a different package it is not automatically updated when you update mythtv itself.
Note that in mythtv-setup there is a button for deleting all channels of a video source. See https://www.mythtv.org/wiki/Channel_Scanning#Delete

Thanks,
Klaas.
Post Reply