Filter out duplicate concurrent programmes

What would you like to see in MythTV and why? Find others who might want to help implement your ideas!
Forum rules
Please be reasonable and positive with your feature requests, remember that all contributions to MythTV are by volunteers in their spare time. MythTV won't support piracy in any form, including torrents and use of soft cams, so to avoid embarrassment please do not ask.

* One suggestion per thread please. Do not post new suggestions in replies. *
Post Reply
TerribleTadpole
Newcomer
Posts: 7
Joined: Wed Mar 29, 2017 9:27 am
Australia

Filter out duplicate concurrent programmes

Post by TerribleTadpole »

Hi all,

I'm thinking I might like to have a swing at this mod myself.

THE ISSUE:
Some networks air the same programme concurrently on different channels. An "any-channel" rule to record the programme results in Myth attempting to record to programme on every channel that it's airing on. The result is a bit of wasted disk space, but more annoyingly it wastes available tuners.

THE CAUSE:
Duplicate filtering currently compares upcoming programmes against previously recorded programmes, but not against upcoming programmes. If a duplicate programme begins before another airing of the same programme has finished then the programme will be recorded again.

SOLUTION IDEA:
Apply the relevant duplicate-prevention option for the recording rule (e.g. title+subtitle) to upcoming recordings and reject any recordings where a duplicate would be created.


I've not worked with Myth before so a bit of guidance with the source would be helpful. I'd like to know:
  • What do you think of the solution idea
  • Where will I find the code that builds the recording schedule (scheduler?)
  • The code that applies the duplicate-prevention option (also scheduler?)
  • Any suggestions you think will be relevant and helpful
Cheers all,
TT
delaytv
Newcomer
Posts: 2
Joined: Wed Apr 05, 2017 8:09 am
United States of America

Re: Filter out duplicate concurrent programmes

Post by delaytv »

There is a schedule filter option for "this channel". Have you tried that?
I had a similar problem. It didn't record the show twice (I only had the one tuner at the time). It would record the show on the std def channel, instead of the high def channel. My guess was because the std def channel number was lower.
User avatar
dizygotheca
Developer
Posts: 267
Joined: Wed Sep 03, 2014 9:02 am
Great Britain

Re: Filter out duplicate concurrent programmes

Post by dizygotheca »

I think your diagnosis is dubious. The scheduler does match duplicates against upcoming recordings.
What I see is 1 recording is marked 'will record' and the following duplicates are marked as 'earlier recording', even if they start before the recording has finished.
If the recording succeeds then the duplicates become 'previously recorded'; if it fails the next duplicate becomes 'will record'.
I've never seen concurrent programmes on different channels but I suspect it will be the same, i.e. it will pick one of them.

It's more likely your issue is generic episodes or different channel/different program ids. https://www.mythtv.org/wiki/Duplicate_matching
Check the program ids and that you don't have duplicate recording rules.

If so, a reliable solution without user intervention is difficult.
An option for the title/episode duplicate matching to override the program id check has been suggested before but it would have to accomodate all domains/EPG sources.
The code is all in the scheduler: https://github.com/MythTV/mythtv/blob/m ... eduler.cpp but would probably impact frontend UI, slave backends, services API etc.
Good luck. It's fragile and not well documented.
TerribleTadpole
Newcomer
Posts: 7
Joined: Wed Mar 29, 2017 9:27 am
Australia

Re: Filter out duplicate concurrent programmes

Post by TerribleTadpole »

Thanks for your input @delaytv and @dizygotheca ...

Using 'this channel' works for things when I know they're coming up. But I have a few generic rules where I want the programmes to be captured regardless of what channel they turn up on.

As mentioned, if the programs are scheduled at different times only one is captured and the other is ignored as a "later showing".

However if the start times are identical then one isn't flagged as recorded before the other begins. The result is that both will start and record to completion. Normally it doesn't matter very much, but because they each take a tuner a third programme can be conflicted out and missed. So I end up with two copies of the one programme and no copies of the other.

As a second observation; I have tuners where a second channel can be demultiplexed out of the same frequency. But when I get the two-at-once scenario each recording takes a second physical tuner rather than using the same physical tuner as the other programme. Doing that would also resolve the conflict problem because the other physical tuner would be available for the other frequency.

I've had a close look at the code. Lots of dynamically generated SQL so it's hard to see exactly what's happening. It looks like we create a temporary table and populate it with the candidates for recording. Then we flag off previously recorded programmes. It's a seriously convoluted piece of thinking that I'm still not across. Luckily I have a database I can grab a snapshot of, that always has some testable cases... :S
User avatar
pgbennett
Developer
Posts: 503
Joined: Mon Apr 27, 2015 5:41 pm
United States of America

Re: Filter out duplicate concurrent programmes

Post by pgbennett »

There must be something different about your listings provider. For me in the USA, with Schedules Direct, MythTV works perfectly to record one and only one copy of every episode. I frequently have the same program available at the same time or at different times and I have never seen it attempt to record more than 1 copy. If a recording fails it will try to record one of the other showings. The exception is local news programs and generic episodes (very few shows have generic episodes). But in those cases it records all showings regardless of whether it has already recorded a showing.

Do your programs have episode ids? You can see by looking at the program details in mythweb. If the episode ids uniquely identify the episode it should work correctly. Perhaps you have no episode ids or inconsistent episode ids.

One way to solve it may be to insert episode ids into the listings xml input.

You could try using schedules direct listings service. I don't know if it is as good everywhere as in the USA but it does have international coverage. I use it in conjunction with tv_grab_zz_sdjson_sqlite (see https://www.mythtv.org/wiki/XMLTV). I believe that tv_grab_zz_sdjson_sqlite has some code that helps with getting good episode ids.

I believe the scheduler code is very complicated. I have never looked at it, but trying to change it may be risky.
perkins1724
Junior
Posts: 33
Joined: Tue Nov 18, 2014 7:44 am
Australia

Re: Filter out duplicate concurrent programmes

Post by perkins1724 »

TerribleTadpole wrote:However if the start times are identical then one isn't flagged as recorded before the other begins. The result is that both will start and record to completion. Normally it doesn't matter very much, but because they each take a tuner a third programme can be conflicted out and missed. So I end up with two copies of the one programme and no copies of the other.
I don't think that is correct. The usual cause is that MythTV is unable to determine that particular airings of a program are actually the same due to listings data being generic. I recommend you first check that MythTV is actually seeing the two programs as the "same" program by checking the programid field and that the programid is not generic (ends in double 0 or triple 0 just working from memory).
TerribleTadpole wrote:As a second observation; I have tuners where a second channel can be demultiplexed out of the same frequency. But when I get the two-at-once scenario each recording takes a second physical tuner rather than using the same physical tuner as the other programme. Doing that would also resolve the conflict problem because the other physical tuner would be available for the other frequency.
Yes, I believe this has occurred for some time and can be frustrating.
mythrec
Newcomer
Posts: 7
Joined: Thu Aug 17, 2017 10:52 am
Great Britain

Re: Filter out duplicate concurrent programmes

Post by mythrec »

Maybe you could consider a python script, you can run it as an event script for the scheduler event. This is from one of the sample scripts in the wiki or something.

Code: Select all

from MythTV import MythBE,MythDB,Channel,Record

def get_recordings_from_backend():
    mythtv = MythBE()

    upcoming = mythtv.getUpcomingRecordings()
    upcoming = list(upcoming) # convert listiterator to list

    print "    found %d upcoming recording(s)" % len(upcoming)

    #figure out the best way to sort / map the programs to pick up the duplicates
    #upcoming = map(convert_program, upcoming)
    #upcoming.sort(sort_programs_by_start)

    for rec in upcoming:
        print rec.title
        print rec.description
You obviously need to add duplicate detection / suppression code. In theory you should be able to do something like the following which would create an override in the scheduler that you can mark as inactive. This code didn't actually seem to work for me but you could always investigating why if it fixes a problem for you.

Code: Select all

#I think these both do the same thing but neither seemed to work for me
#conceptually I think they create a new override in the schedule when you call them
#you're probably supposed to delete the entry if you don't need it
prog=Record(rec.recordid)
prog=Record.fromProgram(rec) 
Post Reply