I want to record duplicate transmissions

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

Post Reply
sdog
Newcomer
Posts: 2
Joined: Sun Aug 21, 2022 9:55 pm
United States of America

I want to record duplicate transmissions

Post by sdog »

Greetings,

I am either dancing all around the solution and missing it, OR there is no solution. Any ideas?

Here is what I'm trying to accomplish: I have several programs that come from multiple transmitters. I live on the edge of reception and depending on the weather (literally - the weather) I may receive a cleaner signal from one transmitter tower than I do the other. I've got ungodly amounts of hard drive space and really want to just record the exact same program from both transmitter towers. If either transmission is clean and successful, great. If one is sketchy, I can watch the other.

Here's my system:

mythtv 32
xubuntu 22.04
us-bcast
ATSC
HDHomerun HDHR5-4US firmware: 20220203
Over the air EIT

Although I receive other transports as well, here's an example of what I'm talking about:

I receive a duplicate multiplexed transport (transport id 481) containing

Fox (KFQX-HD)
CBS (KREY-SD)
Escape (Escape)
Grit (Grit)

from two geographically separate towers, on two separate frequencies, 213MHz (ATSC 13) and 479MHz (ATSC 15). One of these transmissions identifies the multiplex as channel 4 (4_1, 4_2, 4_3, 4_4) the other as channel 13 (13_1, 13_2, 13_3, 13_4).

I'm probably just missing something, but I feel like I've tried every combination of recording setting ("Don't match duplicates") and capture card/video source/input connection/channel edit that I can think of but here's what happens when I try to schedule from the program guide:

Both channels 4 and 13 show up in the program guide. I can select a CBS show from either 4_2 or 13_2 and program it to record. As I return to the program guide after the schedule save, BOTH identical programs are highlighted. But when I go to upcoming recordings or the media library, only one transmission is actually recording. I'm guessing that mythtv/HDHR is making a decision about which transmission is "better" and recording only that transmission. There may be a way to see it, but I can't tell WHICH of the two frequencies is actually being recorded. And I can't figure out how to record BOTH frequencies.

Here are a few of the things I've tried twiddling with so far:

Recording settings: played with "don't match duplicates" setting. No love

Capture cards: no experimentation here. Just four cards representing the four tuners in the HDHR. (I CAN record "multiple" programs on this device just fine by the way, just not "duplicate" programs.) I don't think there's much else I could change here.

Video Sources: It seems in theory, because of how the HDHR and over the air EIT works, that I should just use a single Video Source containing all the channels, so that's mostly what I tried. But I also experimented with creating several Video Sources, one containing channel 4, the other channel 13.

Input Connections: Default setup: Created four, one for each capture card, each pointing to the one same video source. Experimental setup: When I tried the multiple video sources above, I also tried attaching one card to the second video source. Did not try any of the settings for "schedule as group" or "create new input group" because I could not tell from the documentation exactly what input group configuration might help my attempts. I will go there if you tell me that is the direction to head.

Channel Scan/Edit: Scanned with the setting to allow duplicates. All expected transports/frequencies/channels are in the list. All can be watched and recorded individually. When I tried the multiple video source experiment, obviously there were duplicate channels in the list, but the program guide still distilled duplicate channel choices to single channel entries.

What I DIDN'T try is jiggering one of the channel's "channel name" or "callsign" settings to make it unique from it's duplicate. Do you think that would make the system see it as two separate programs, or would that just screw up the EIT retrieval?

What other settings combinations have I missed that might allow me to record copies of a program from two separate transmitters?

Thanks in advance for any help you can offer.

Doug
User avatar
kmdewaal
Developer
Posts: 644
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: I want to record duplicate transmissions

Post by kmdewaal »

TL;DR I am not the expert on this but I think that MythTV currently just cannot do what you want. I suggest to use GIt and create a Feature Request issue (https://github.com/MythTV/mythtv/issues). Reason is that not everybody reads the forum and and the request will not get lost. It might be rejected but at least somebody will have looked at it.

This is how I think that it works.

All the Duplicate Matching that can be configured does only match new programs to be recorded against existing recordings. It does NOT match programs to be recorded against each other.

Recordings are identified by the start time plus program title+description. The callsign, the channel name or the channel ID do not matter.
If there is only one character different in for example the description then it are different recordings. If the start time is only one minute different then it is a different recording. But if start time plus program title+description is exactly the same then only one recording will be made.

The identification by start time plus program title+description is also what is done in the program guide. Therefore you will see the program that is scheduled to be recorded on all channels where it can be recorded.

When it comes to making the recording and if the same recording can be done on different channels, then it is more or less random which channel will be selected but only one recording will be made. There is no preference of channel or multiplex based on reception quality or so.
You can configure a preference for a channel by giving it a priority and then that channel will be used. This is however completely static; once configured it will always prefer that channel.

The channel scan of mythtv-setup does remove duplicate multiplexes when multiplexes can be received from multiple transmitters, based on signal strength or signal quality. This is however done only for multiplexes that are completely identical. In your case the channels are identical but the multiplexes are not identical because channel 4 is not the same as channel 13.

So there is, as I understand it, not a configuration option for MythTV to make duplicate recordings of identical programs.

About possible hacks....
If the start time of a program is changed by one minute then it would be a different program. If a string is appended to the description then it would also be a different program. This does of course require database programming skills. Also, the EIT will overwrite all changes made.
To fix that you would need to split your channels in two video sources, use EIT for one but not for the other, and write something that copies the EIT data from one video source to the other and then apply the fixup to start time or description to make the recording unique.
This is not trivial.

Hope this helps,
Klaas.
sdog
Newcomer
Posts: 2
Joined: Sun Aug 21, 2022 9:55 pm
United States of America

Re: I want to record duplicate transmissions

Post by sdog »

Klaas, Your response is EXTREMELY helpful to my understanding of how this works. Not sure which direction I'll take it from here, but I feel like I have a much better sense of what is possible and what is not.

Thank You!!!!!
Doug
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: I want to record duplicate transmissions

Post by paulh »

David E. (gigem) would be the man to ask. He is on the forum but it's very rare we see him on here helping users :cry:

Like kmdewaal the scheduler is a complete mystery to me so I may be talking complete BS. I think a lot of what kmdewaal said is probably true if you only use EIT for EPG data but if you use something like Schedules Direct which have proper programid's for most/all programs then they I believe they will always take precedence over the program Title, Subtitle and Description as far as duplicate detection goes. Since most users don't want duplicates, some get very angry when they do :evil:, MythTV by default will not record duplicates. I wonder if you could somehow trick MythTV to record them anyway? As far as the scheduler goes any channel with the same Callsign is assumed to be the same station/channel containing the same set of programs so normally the scheduler will choose to record from one of the stations/channels that have that callsign and mark all the others as duplicates so I think the first thing you would have to do is tweak the callsigns so they are different. Even then I'm not sure if it would work even if you turn off all the duplicate detection so it may still need some code changes.
User avatar
kmdewaal
Developer
Posts: 644
Joined: Wed Dec 07, 2016 8:01 pm
Netherlands

Re: I want to record duplicate transmissions

Post by kmdewaal »

I agree with paulh that gigem is the one who know this.

But on the topic of programid. They do indeed take precedence over title/subtitle/description when they are present, but the duplicate detection is done only for programs that have already been recorded as I recently found out.

Use case: program "Vintage Voltage" on UK Freesat (satellite Astra 2 on 28.2E) channels Quest and Quest HD. These two channels have exactly the same programs, only one channel is SD and the other is HD as you might expect from the channel name.
The programid value of the program on both channels, received with EIT, is the same.
The only thing that is different is the description; the SD version has SxEx (season/episode) added and the HD version has not.
This difference is enough to create duplicate recordings, every day at 12:00.

My conclusion for now is that A. duplicate matching is done for existing recordings only and B. the channel name/callsign does not matter.
Whether this can be considered a bug or a feature I have not yet figured out.

However, this makes me think of possible solutions. If the channels to be recorded would be divided in two different video sources and there is an EIT fixup (n.b. this is code change) that appends the name of the video source to the description then a duplicate recording would be made.

Note that to use two different video sources one does need in this case two HDHomeRun devices.
Which is a nice bridge to the topic of how to receive signals from two different transmitters.
I think the best way to do this is to have two separate antenna's, each pointed to one transmitter.
Then each antenna its own HDHomeRun device.
Each HDHomeRun device its own video source.
Do a channel scan for each video source.
Select for each channel that can be received on both video sources the one that has the worst reception and mark that channel as invisible.
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: I want to record duplicate transmissions

Post by PhilB »

Would issuing an AddRecordSchedule API call when a recording starts but with changed channel number and callsign and a start time delayed by a few seconds work?
This does something similar: viewtopic.php?f=19&t=4110
Phil
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: I want to record duplicate transmissions

Post by PhilB »

Hm. Amazingly that seems to work. These APIs are so powerful!

I have pairs of channels here in Uk which usually transmit the same content, one in SD the other in HD.
BBC One and BBC One HD for example.

If I manually trigger a recording for one in the frontend guide, the 'upcoming recordings' shows one recording but in the guide the other one is marked as also being recorded - whether shown in the frontend or via an API call.

However, if you force the 'other' one by copying the rule and changing CallSign and ChanId and inserting 'Station' to an API call it appears in 'upcoming recordings' and gets recorded.

Caveats:
1. my station names/callsigns are different. Yours may not be.
2. It may be necessary to add more entries in the $recrule{something} in your TV numbering culture.
3. The table gives source -> target channel mapping. Avoid recursion!!


Here is the demo code: Save it, make it executable then ./xmit2.pl --help for details.
Have fun!
Phil

Code: Select all

#!/usr/bin/perl -w
#
#  Duplicate recordings across 2 transmitters
#

use strict;
use Getopt::Long;
use Time::Local;
use Fcntl qw(:flock);

#
#  User setup
#  ----------

my $lockfile='/home/mythtv/.mythtv/xmit.lock'; 
my $backend='http://127.0.0.1:6544';
# List of interesting channels eg BBC one => BBC one HD.
my %target=(
    20001 => 20101,
);
# -----------------
# End of user setup
# -----------------


my $content;
my %guide;

#Get calling params
my $calling=join(' ',@ARGV);
			
my $help=0; my $listchannels=0;  my $rule=-1;
GetOptions ('list'=>\$listchannels,'help' => \$help,
			'rule=i' => \$rule);

givehelp() if ($help);


#we need a module:
BEGIN {   
    unless (eval "require scan_database") {
        print "couldn't load scan_database module\nSee https://www.mythtv.org/wiki/Perl_API_examples\n";
    }
}
listchannels() if ($listchannels);

#Hash of recording status texts and whether they indicate records is being/will be made
my %recording=(
    WillRecord => 1,
    Pending    => 1,
    Tuning     => 1,
    Recording  => 1,
    Unknown    => 0,
    Failed     => 0,
    Recorded   => 0
);

if ($rule<0){print "No rule specified\n"; exit 0;};

#open log file and lock it
open(LOCKFILE,'>',$lockfile) or die "Cannot open logfile $lockfile:\n$!\n";
flock(LOCKFILE, LOCK_EX);     #wait til it's free
my $now=TimeString(time());
print "\n$now xmit $calling\n";


#get recording rule

my $url=$backend . "/Dvr/GetRecordSchedule?RecordId=$rule";
scan_database::ReadBackend($url, $content);
my %recrule;
scan_database::GetAllFields(%recrule, $content, '>', '</RecRule>');

unless (exists $target{$recrule{ChanId}}) {myexit("Not an interesting channel")};
print "Interesting Channel: $recrule{ChanId}:  $recrule{CallSign}\n";

my $targetchan=$target{$recrule{ChanId}};
if (exists $target{$targetchan}){ myexit("Faulty config - target $targetchan cannot be a source")};

#get guide data for target
print "Searching target channel $targetchan for '$recrule{Title}'\n";
&getguide($targetchan,$now);
&showguide();

#look for matching program in guide
my $index=-1;
for (0..1){
	if (exists $guide{$_}){
		if ($guide{$_}{ProgramId} eq $recrule{ProgramId}){
			$index=$_;   #found match
		};
	}
}
if ($index < 0){myexit('Cannot find matching program')};

#Target may always be marked as 'Recording' so omit this check 
#if ($recording{$guide{$index}{Status}} ==1){
#	print "already recording\n";
#	myexit('Target is already being recorded')
#};


#Adjust recording rule:
$recrule{CallSign}=$guide{$index}{CallSign};
$recrule{Station}=$guide{$index}{CallSign};  #documentation demands this (why?)
$recrule{ChanId}=$guide{$index}{ChanId};
# NB in USA you amy need to change more!


#Issue recording rule
print "Inserting rule\n";
scan_database::ValidatePost(%recrule, $backend .'/Dvr/AddRecordSchedule', 'raw', 12);

#Don't unlock and allow another invocation until database has had time to update
sleep 10;   #should be enough
myexit('Done');


sub getguide{
	my ($chan, $start)=@_;

	#Read the guide, show callsign, get 6 entries and show them
	my $url="$backend/Guide/GetProgramList?StartTime=$start&ChanId=$chan&Count=2&Details=true";
	unless (scan_database::ReadBackend($url,$content)){myexit("Could not get guide data")};
		
	#Did we get any guide data?
	$content =~ m!<Count>(\w+)</Count>!;
	myexit('No guide data') if ($1==0);
	#extract fields
	my @guideitems=qw/StartTime EndTime Title ProgramId Status Description ChanId CallSign/;
	scan_database::FillHashofHash(%guide, $content,'Program','#', @guideitems);
	
	#check valid status values
	for (0..1){
		my $status=$guide{$_}{Status};
		unless (exists $recording{$status}){
			print "warning:  Status not known: $status\n"; 
			$recording{$status}=1;   #If we know no better then don't allow to re-record
		}
	}
}

sub showguide{
	print "# Status       StartTime             Title\n";
	for (0..1){
		printf "$_ %-10s %22s  $guide{$_}{Title}\n", $guide{$_}{Status}, $guide{$_}{StartTime};
	};
}
	
	
sub myexit{
	if ($_[0] ne ''){print "$_[0]\n"};
	close(LOCKFILE);
	exit 0;
}

sub TimeString{
	(my $epoch)=@_;
	#return time as 2021-12-03T13:44:04
	my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = gmtime($epoch);
	$year+=1900; $mon++;
	return sprintf("%04d-%02d-%02dT%02d:%02d:%02dZ", $year, $mon, $mday, $hour, $min, $sec);
}

	
sub listchannels{
	#list all channels 
	my $temp;  my %sources; my %ChanData;
	#get sources
	my $url=$backend. '/Channel/GetVideoSourceList';
	scan_database::ReadBackend($url, $temp);
	scan_database::FillHashofHash(%sources, $temp, 'VideoSource', 'Id', 'SourceName');
    #get channels per source
    for my $source (keys %sources){
		scan_database::ReadBackend($backend . '/Channel/GetChannelInfoList?SourceID='.$source.
					'&OnlyVisible=false&Details=true', $temp);
		my %temphash;
		scan_database::FillHashofHash(%temphash, $temp, 'ChannelInfo', 'ChanId', 'CallSign','Visible');
		%ChanData = (%ChanData, %temphash);
	}
    for (sort keys %ChanData){
		if ($ChanData{$_}{Visible} eq 'true'){print "$_ = $ChanData{$_}{CallSign}\n"};
	}
	exit 0; 
}

sub givehelp{

my $logfile='/var/log/mythtv/xmit.log';
my $location='/usr/local/bin';
print "
    
xmit2.pl
============
    
Aims
----
Trigger a second recording for a channel duplicated across two transmitters.  This is useful in a 
scenario where transmissions are unreliable and one may be better than the other on different days.

How does it work?
-----------------
It will take recordings from a 'source' channels and duplicate them for 'target' channels. 
 
The script is run by a system event at the start of a recording.

It will extract the recording rule and after checking that the source channel is an 'interesting' 
one, it will read the first 2 programs (#0 and #1) for the 'target' channel from the guide

If either of the first two entries in the guide has the same ProgramId as in the 'source' recording then it will
create a new recording rule for it.

Note that the two starting entries in the guide are checked because the script may be run:
- before scheduled start time if pre-scheduled (source will be #1) or
- after if manually triggered after the source recording started (source will be #0).



Parameters
----------
   --rule          eg --rule 5565
   --help or -h    this text
   --list or -l    list all visible channels with chanid and callsign.
 

Setup (assumes Ubuntu)
----------------------
1.  Put this script in (say) $location/xmit2.pl and make it executable (chmod +x).
2.  Put the module scan_database.pm in perl path and make it everyone readable.  
    See:  https://www.mythtv.org/wiki/Perl_API_examples
3.  Create a log file and a lock file:
    sudo touch $logfile
    sudo chmod 644 $logfile
    sudo touch $lockfile
    sudo chown mythtv:mythtv $lockfile
    sudo chmod 644 $lockfile
4.  Set up a system event Recording Started:
    ${location}/xmit2.pl --rule %RECID% >> $logfile 2>&1
    

Channel Filter
--------------
You will need to change the hash \%target to match source and target channel numbers.
As issued, a recording of BBC One with channel ID of 20001 will trigger a recording on
and BBC One HD with an id of 20101.
Establish these ChanId values with --list.


Logging
-------
Logging is maintained by a redirect in the system event line.

Locking
-------
To prevent problems with two sumultaneous invocations of the code a lock file is used: $lockfile

Times
-----
Note that all times are in UTC.

Phil Brady.  31 August2022.
";
exit 0;
}
PhilB
Senior
Posts: 403
Joined: Sun May 11, 2014 6:23 pm
Great Britain

Re: I want to record duplicate transmissions

Post by PhilB »

I have been pondering this behaviour and testing it with BBC one and BBC One HD on a v32 backend.
When a mythtv frontend (one with database access) attempts to trigger duplicate recordings the scheduler will prevent it.
With a program using API post calls the scheduler allows it.
Similarly, if the two recordings are triggered on a Leanback Frontend (in my case on a Firestick) which uses the API interface then they both get recorded.
Now is this a difference in the recording settings or does the API call just miss out the scheduler check?
Either way, it may be another answer to the original poster.
Phil
Post Reply