Script to export Mythtv recordings to a folder or NAS

For discussion related to MythTV which doesn't belong in another forum.

Moderator: Forum Moderators

skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Script to export Mythtv recordings to a folder or NAS

Post by skinah »

Hi All,

I've written these scripts and wish to share them with others and would welcome any improvements that people suggest. There are lots of other scripts around but none of them fully met what I was trying to achieve, this was....

1. Leave mythTV recordings fully untouched so if a mistake in a cutlist was found, I could rerun the script to fix issues.
2. Create a folder and name structure that is Kodi compatible.
3. Compress the file down to a mp4 file so the raspberry pi2 does not need a mp2 licence for decoding, remove interlacing and to drop the file size down.
4. If the NAS was offline when the script was running it leaves the file already compressed in a fallback folder to save the time that handbrake takes to run.
5. Be written 100% in shell commands and kept as simple as possible with comments to help others modify it to their needs. Just comment out any lines you wish to skip.

You do not need a NAS, you can use this to export to MP4 in a friendly name format to any folder. It does not touch your database or replace the mythtv recordings with lower quality.

Why two scripts?
I run one script for TV shows because some shows would end up with the same file name if treated the same way as a movie. The TV Show script will add the recording time stamp after the shows name if it is not part of a known series. An example is 'Todays News - 20160324102800.mp4'
This way if you record a show that has the same name everyday it will keep them all and they will not get overwritten as the timestamp is in the filename.
The movie script will output a file like 'Shrek.mp4' with no time stamp so if you record the same movie you will overwrite the old one and not end up with multiple copies. Both scripts will handle TV shows that will output as 'Lost - s1e1.mp4' and either script can be used.

I would like to merge both into the same script but currently I can not see a way to do this without reading the database directly or having a new feature added to mythtv.

TV Shows script

Code: Select all

#!/bin/sh
# Script written by 'Skinah'.
# This script will export a mythtv recording to a folder in a friendly readable name that is 
# compatible with KODI. It will honor cutlists and also use handbrake to compress recordings
# to MP4 to save HDD space.  
#
# It now supports lossless cutting of MP4 HD footage as well as MPEG2. 
# All of this is done without changing Mythtv's database or recordings, this way if a 
# mistake is found in the cutting of the recording then before myth expires the recording, we can 
# rerun the job after adjusting the cutlist manually.
#
############### To install do the following #######################
# 1. Add this next line to the mythbackend setup for a user job
# /pathtoscript/TVShowToNAS.sh "%CHANID%" "%STARTTIMEUTC%" "%TITLE%" "%SUBTITLE%" "%SEASON%" "%EPISODE%" "%FILE%" "%DIR%"
#
# 2. Save this as a plain text file with .sh extention making sure it matches what you used in the last step.
# 3. Set the file to be executable.
# 4. Setup the script by editing the below text.
# 5. Install handbrake and any other requirements for MP4 source recordings.
# 
# to install handbrake use these commands or visit the handbrake website.
# sudo add-apt-repository ppa:stebbins/handbrake-releases
# sudo apt-get update
# sudo apt-get install handbrake-gtk handbrake-cli
#
# To do lossless cutting with MP4 recordings do the following to install the requirements. MP2 does not need these.
# Visit https://mkvtoolnix.download/downloads.html and add the software sources for your linux build and the signed key.
# Any problems with mp4 cuts could be caused by an old version of mkvtoolnix so make sure you follow the above line.
# sudo apt-get install mkvtoolnix mkvtoolnix-gui
# sudo apt-get install bc
############# Section for Setting up script and changing options ########################

# path to MythTV transcoding tools, tested with mythbuntu 14.04
INSTALLPREFIX=/usr/bin

#If your HD channel is not 25 frames/sec then edit this with the value. Only used for h264 cuts.
fps=25

#Quality Settings, a value of 0 will give lossless cuts and skip handbrake encoding. 
#A lower number will mean higher video quality and larger file size.
TVShowQuality=25

# Where your finished videos will be placed. This can be a NAS drive that is mounted in etc/fstab
# Mythtv user must have write permissions for this location.
# If you want movies and tv shows to go to the same place, just make both of these the same.
MOVIEOUTDIR="/mnt/nas/Movies"
TVOUTDIR="/mnt/nas/TV Shows"

# If the move command fails due to the NAS not being mounted, the file will remain here so as not to waste the Handbrake time.
# Mythtv user must have write permissions for this location. 
# If you do not have a NAS, think of this as a working aka temp folder.
FALLBACKDIR="/home/matt/Videos"

######## Leave everything below alone unless you know how to mod and debug #############

# Store the variables that the userjob passes on to this shell script
TVTITLE=$3
TVSUBTITLE=$4
SeasonEpisode="s$5e$6"
MYTHRECORDINGDIR=$8

# First clean up any weird tags that the TV channel adds.
# These helped when Shepperd was not setup for XMLTV in Australia.
TVTITLE=`echo "$TVTITLE" | sed 's/Movie Premiere: //g' | sed 's/- Season Premiere//g' | sed 's/Movie: //g' | sed 's/://g' | sed 's/?//g' | sed 's/,//g'`
TVSUBTITLE=`echo "$TVSUBTITLE" | sed 's/\[RPT\] //g' | sed 's/(R) //g' | sed 's/ - //g' | sed 's/(G) //g' | sed 's/(AV) //g' | sed 's/(M) //g' | sed 's/(PG) //g' | sed 's/\[CC\] //g' | sed 's/(MA) //g'`

# clean up the title and subtitle strings by deleting any non-standard characters that can cause issues in file names
TVTITLE=`echo "$TVTITLE" | sed 's/://g' | sed 's/?//g' | sed 's/,//g' | sed 's/\///g'`
TVSUBTITLE=`echo "$TVSUBTITLE" | sed 's/://g' | sed 's/?//g' | sed 's/,//g' | sed 's/\///g'`

# print the following for logging purposes
echo "The line below records the file details of the last run." > $FALLBACKDIR/ScriptsLastRun.log
echo "ChanID_StartTimeUTC-Title-SeasonAndEpisodeNumber-Subtitle" >> $FALLBACKDIR/ScriptsLastRun.log
echo "$1_$2-$TVTITLE-$SeasonEpisode-$TVSUBTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
echo "DIR/Filename" >> $FALLBACKDIR/ScriptsLastRun.log
echo "$8/$7" >> $FALLBACKDIR/ScriptsLastRun.log

# play nice with other processes
MYPID=$$
renice 19 $MYPID
ionice -c 3 -p $MYPID

#Check to see if we already have a cutlist before we overwrite any manually created skips, otherwise create new cutlist.
if [ -z "`$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist | grep Cutlist | sed 's/Cutlist: $//'`" ]; then
    echo "no cutlist found....generating new cutlist"
    $INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --gencutlist
fi  

$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist >> $FALLBACKDIR/ScriptsLastRun.log

# Try to remove the cutlist without transcoding so as not to lose quality. This fails on MP4 files as its not yet supported by mythtranscode so we must use another method.
echo "Using mythtranscode to remove cutllist after first creating a backup." >> $FALLBACKDIR/ScriptsLastRun.log

#backup recording as a workaround to a bug in 0.28
#cp $MYTHRECORDINGDIR/$7 $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log

# Remove the cutlist without transcoding so as not to lose quality.
$INSTALLPREFIX/mythtranscode --chanid "$1" --starttime "$2" --mpeg2 --honorcutlist -o "$FALLBACKDIR/$TVTITLE-$2.mpg" >> $FALLBACKDIR/ScriptsLastRun.log

############## Work out if MP4 or MP2 code needs to run ###############

#check exit code
if [ $? = 0 ]
then
	echo "Lossless transcode on MP2 using mythtranscode worked fine" >> $FALLBACKDIR/ScriptsLastRun.log
	# next two lines are for a work around to a bug.
	#cp $FALLBACKDIR/$1_$2.bup $MYTHRECORDINGDIR/$7 >> $FALLBACKDIR/ScriptsLastRun.log
	#rm $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log
else
	#rm $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log
    	echo "File must be a mp4 recording as we can not do lossless transcoding with mythtranscode" >> $FALLBACKDIR/ScriptsLastRun.log

CUTLIST=`$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist | grep Cutlist | sed 's/Cutlist: //' | \
    sed 's/-/ /g' | sed 's/^\|$/-/g' | sed 's/,/-/g'`

echo "The cutlist is:" >> $FALLBACKDIR/ScriptsLastRun.log
echo $CUTLIST >> $FALLBACKDIR/ScriptsLastRun.log

keepframes=""
keepsecond=""
clipcount=0
for i in ${CUTLIST}
do
    start=`echo $i | sed 's/ //g' | sed 's/^\(.*\)-.*$/\1/'`
    end=`echo $i | sed 's/ //g' | sed 's/^.*-\(.*\)$/\1/'`

    	#if $start is empty, deal with it
    	if [ -z $start ]; then
	start=0
	fi

    	#If the end is empty then guess how long the file is.
	if [ -z $end ]; then
	end=500000
	fi
	
	#if the start and end are the same then skip the cut
	if [ $start -eq $end ]; then
	continue 
	fi

	#if the clipcount is greater than 0 we need to add ,+
	if [ $clipcount -gt 0 ]; then
	keepframes="$keepframes,+"
	keepsecond="$keepsecond,+"
	fi
    
    	
	keepframes="$keepframes$start-$end"

	clipcount=$((clipcount+1))

	echo "clipcount=$clipcount" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Using frames:" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Start=$start-End=$end" >> $FALLBACKDIR/ScriptsLastRun.log

	#convert start into time in seconds (divide frames by frames per second)    
    	start=$(echo "scale=2; $start / $fps" | bc -l)
	#Use the program bc to create a floating point value in seconds with 2 decimal places.
	end=$(echo "scale=2; $end / $fps" | bc -l)
	echo "Using Seconds:" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Start=$start-End=$end" >> $FALLBACKDIR/ScriptsLastRun.log

	keepsecond="$keepsecond$(date -d@$start -u +%H:%M:%S)-"
	keepsecond="$keepsecond$(date -d@$end -u +%H:%M:%S)"

done

echo "The keepframes string contains:$keepframes" >> $FALLBACKDIR/ScriptsLastRun.log
echo "The keepsecond string contains:$keepsecond" >> $FALLBACKDIR/ScriptsLastRun.log
echo "Using mkvmerge to create a single file without pieces from the cutlist" >> $FALLBACKDIR/ScriptsLastRun.log

mkvmerge --split parts:$keepsecond -o "$FALLBACKDIR/$TVTITLE-$2.mkv" "$MYTHRECORDINGDIR/$7" >> $FALLBACKDIR/ScriptsLastRun.log
#mkvmerge --split parts-frames:$keepframes -o "$FALLBACKDIR/$TVTITLE-$2.mkv" "$MYTHRECORDINGDIR/$7" >> $FALLBACKDIR/ScriptsLastRun.log

if [ "$SeasonEpisode" = "s0e0" ] && ["$TVSUBTITLE" = ""]
then
        mkdir -p "$TVOUTDIR/$TVTITLE"

	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mkv" "$TVOUTDIR/$TVTITLE/$TVTITLE - $2.mkv"
	else
	        /usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mkv" -o "$FALLBACKDIR/$TVTITLE - $2.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4
		mv -u "$FALLBACKDIR/$TVTITLE - $2.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $2.m4v"
		rm "$FALLBACKDIR/$TVTITLE-$2.mkv"
	fi

	rm "$FALLBACKDIR/$TVTITLE-$2.mpg"
	exit 0
else
        mkdir -p "$TVOUTDIR/$TVTITLE"
	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mkv" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mkv"
	else
		/usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mkv" -o "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v"
		rm "$FALLBACKDIR/$TVTITLE-$2.mkv"
	fi
	rm "$FALLBACKDIR/$TVTITLE-$2.mpg"
	exit 0
fi



fi
######## End of MP4 only code ############


if [ "$SeasonEpisode" = "s0e0" ] && ["$TVSUBTITLE" = ""]
then
	mkdir -p "$TVOUTDIR/$TVTITLE"
	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mpg" "$TVOUTDIR/$TVTITLE/$TVTITLE - $2.mpg"
	else
	        /usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mpg" -o "$FALLBACKDIR/$TVTITLE - $2.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4
        	mv -u "$FALLBACKDIR/$TVTITLE - $2.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $2.m4v"
		rm "$FALLBACKDIR/$TVTITLE-$2.mpg"
	fi
	rm "$FALLBACKDIR/$TVTITLE-$2.mpg.map"
	exit 0
else
        mkdir -p "$TVOUTDIR/$TVTITLE"
	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mpg" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mpg"
	else
		/usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mpg" -o "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v"
		rm "$FALLBACKDIR/$TVTITLE-$2.mpg"
	fi
	rm "$FALLBACKDIR/$TVTITLE-$2.mpg.map"
	exit 0
fi



This following script is for Movies. If it is run by mistake on a tv show, it will detect this and still handle tv shows that are part of a known series. It has better error logging than the previous script currently so get this one working first.

Code: Select all

#!/bin/sh
# Script written by 'Skinah'.
# This script will export a mythtv recording to a folder in a friendly readable name that is 
# compatible with KODI. It will honor cutlists and also use handbrake to compress recordings
# to MP4 to save HDD space.  
#
# It now supports lossless cutting of MP4 HD footage as well as MPEG2. 
# All of this is done without changing Mythtv's database or recordings, this way if a 
# mistake is found in the cutting of the recording then before myth expires the recording, we can 
# rerun the job after adjusting the cutlist manually.
#
############### To install do the following #######################
# 1. Add this next line to the mythbackend setup for a user job
# /pathtoscript/sendtonas.sh "%CHANID%" "%STARTTIMEUTC%" "%TITLE%" "%SUBTITLE%" "%SEASON%" "%EPISODE%" "%FILE%" "%DIR%"
#
# 2. Save this as a plain text file with .sh extention making sure it matches what you used in the last step.
# 3. Set the file to be executable.
# 4. Setup the script by editing the below text.
# 5. Install handbrake and any other requirements for MP4 source recordings.
# 
# to install handbrake use these commands or visit the handbrake website.
# sudo add-apt-repository ppa:stebbins/handbrake-releases
# sudo apt-get update
# sudo apt-get install handbrake-gtk handbrake-cli
#
# To do lossless cutting with MP4 recordings do the following to install the requirements. MP2 does not need these.
# Visit https://mkvtoolnix.download/downloads.html and add the software sources for your linux build and the signed key.
# Any problems with mp4 cuts could be caused by an old version of mkvtoolnix so make sure you follow the above line.
# sudo apt-get install mkvtoolnix mkvtoolnix-gui
# sudo apt-get install bc
############# Section for Setting up script and changing options ########################

# path to MythTV transcoding tools, tested with mythbuntu 14.04
INSTALLPREFIX=/usr/bin

#If your HD channel is not 25 frames/sec then edit this with the value. Only used for h264 cuts.
fps=25

#Quality Settings, a value of 0 will give lossless cuts and skip handbrake encoding. 
#NOTE: If it is a movie and from a MP4 source it will be lossless regardless of this setting. 
#A lower number will mean higher video quality and larger file size.
MovieQuality=21
TVShowQuality=25

# Where your finished videos will be placed. This can be a NAS drive that is mounted in etc/fstab
# Mythtv user must have write permissions for this location.
# If you want movies and tv shows to go to the same place, just make both of these the same.
MOVIEOUTDIR="/mnt/nas/Movies"
TVOUTDIR="/mnt/nas/TV Shows"

# If the move command fails due to the NAS not being mounted, the file will remain here so as not to waste the Handbrake time.
# Mythtv user must have write permissions for this location. 
# If you do not have a NAS, think of this as a working aka temp folder.
FALLBACKDIR="/home/matt/Videos"

######## Leave everything below alone unless you know how to mod and debug #############

# Store the variables that the userjob passes on to this shell script
TVTITLE=$3
TVSUBTITLE=$4
SeasonEpisode="s$5e$6"
MYTHRECORDINGDIR=$8

# First clean up any weird tags that the TV channel adds.
# These help if you do not use a XMLTV grabber like Shepperd and use IET for guide data.
TVTITLE=`echo "$TVTITLE" | sed 's/Movie Premiere: //g' | sed 's/- Season Premiere//g' | sed 's/Movie: //g' | sed 's/://g' | sed 's/?//g' | sed 's/,//g'`
TVSUBTITLE=`echo "$TVSUBTITLE" | sed 's/\[RPT\] //g' | sed 's/(R) //g' | sed 's/ - //g' | sed 's/(G) //g' | sed 's/(AV) //g' | sed 's/(M) //g' | sed 's/(PG) //g' | sed 's/\[CC\] //g' | sed 's/(MA) //g'`

# clean up the title and subtitle strings by deleting any non-standard characters that can cause issues in file names
TVTITLE=`echo "$TVTITLE" | sed 's/://g' | sed 's/?//g' | sed 's/,//g' | sed 's/\///g'`
TVSUBTITLE=`echo "$TVSUBTITLE" | sed 's/://g' | sed 's/?//g' | sed 's/,//g' | sed 's/\///g'`

# print the following for logging purposes
echo "The line below records the file details of the last run." > $FALLBACKDIR/ScriptsLastRun.log
echo "ChanID_StartTimeUTC-Title-SeasonAndEpisodeNumber-Subtitle" >> $FALLBACKDIR/ScriptsLastRun.log
echo "$1_$2-$TVTITLE-$SeasonEpisode-$TVSUBTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
echo "DIR/Filename" >> $FALLBACKDIR/ScriptsLastRun.log
echo "$8/$7" >> $FALLBACKDIR/ScriptsLastRun.log

# play nice with other processes
MYPID=$$
renice 19 $MYPID
ionice -c 3 -p $$

#Check to see if we already have a cutlist before we overwrite any manually created skips, otherwise create new cutlist.
if [ -z "`$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist | grep Cutlist | sed 's/Cutlist: $//'`" ]; then
    echo "no cutlist found....generating new cutlist" >> $FALLBACKDIR/ScriptsLastRun.log
    $INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --gencutlist >> $FALLBACKDIR/ScriptsLastRun.log
fi  


$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist >> $FALLBACKDIR/ScriptsLastRun.log

# Try to remove the cutlist without transcoding so as not to lose quality. This fails on MP4 files as its not yet supported by mythtranscode so we must use another method.
echo "Using mythtranscode to remove cutllist after first creating a backup." >> $FALLBACKDIR/ScriptsLastRun.log

#backup recording as a workaround to a bug in 0.28
#cp $MYTHRECORDINGDIR/$7 $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log
$INSTALLPREFIX/mythtranscode --chanid "$1" --starttime "$2" --mpeg2 --honorcutlist -o "$FALLBACKDIR/$TVTITLE-$2.mpg" >> $FALLBACKDIR/ScriptsLastRun.log


############## Work out if MP4 or MP2 code needs to run ###############

#check exit code
if [ $? = 0 ]
then
	echo "Lossless transcode on MP2 using mythtranscode worked fine" >> $FALLBACKDIR/ScriptsLastRun.log
	# next two lines are for a work around to a bug.
	#cp $FALLBACKDIR/$1_$2.bup $MYTHRECORDINGDIR/$7 >> $FALLBACKDIR/ScriptsLastRun.log
	#rm $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log
else
	#rm $FALLBACKDIR/$1_$2.bup >> $FALLBACKDIR/ScriptsLastRun.log
    	echo "File must be a mp4 recording as we can not do lossless transcoding with mythtranscode" >> $FALLBACKDIR/ScriptsLastRun.log

CUTLIST=`$INSTALLPREFIX/mythutil --chanid "$1" --starttime "$2" --getcutlist | grep Cutlist | sed 's/Cutlist: //' | \
    sed 's/-/ /g' | sed 's/^\|$/-/g' | sed 's/,/-/g'`

echo "The cutlist is:" >> $FALLBACKDIR/ScriptsLastRun.log
echo $CUTLIST >> $FALLBACKDIR/ScriptsLastRun.log

keepframes=""
keepsecond=""
clipcount=0
for i in ${CUTLIST}
do
    start=`echo $i | sed 's/ //g' | sed 's/^\(.*\)-.*$/\1/'`
    end=`echo $i | sed 's/ //g' | sed 's/^.*-\(.*\)$/\1/'`

    	#if $start is empty, deal with it
    	if [ -z $start ]; then
	start=0
	fi

    	#If the end is empty then guess how long the file is.
	if [ -z $end ]; then
	end=500000
	fi
	
	#if the start and end are the same then skip the cut
	if [ $start -eq $end ]; then
	continue 
	fi

	#if the clipcount is greater than 0 we need to add ,+
	if [ $clipcount -gt 0 ]; then
	keepframes="$keepframes,+"
	keepsecond="$keepsecond,+"
	fi
    
    	
	keepframes="$keepframes$start-$end"

	clipcount=$((clipcount+1))

	echo "clipcount=$clipcount" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Using frames:" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Start=$start-End=$end" >> $FALLBACKDIR/ScriptsLastRun.log

	#convert start into time in seconds (divide frames by frames per second)    
    	start=$(echo "scale=2; $start / $fps" | bc -l)
	#Use the program bc to create a floating point value in seconds with 2 decimal places.
	end=$(echo "scale=2; $end / $fps" | bc -l)
	echo "Using Seconds:" >> $FALLBACKDIR/ScriptsLastRun.log
	echo "Start=$start-End=$end" >> $FALLBACKDIR/ScriptsLastRun.log

	keepsecond="$keepsecond$(date -d@$start -u +%H:%M:%S)-"
	keepsecond="$keepsecond$(date -d@$end -u +%H:%M:%S)"

done

echo "The keepframes string contains:$keepframes" >> $FALLBACKDIR/ScriptsLastRun.log
echo "The keepsecond string contains:$keepsecond" >> $FALLBACKDIR/ScriptsLastRun.log
echo "Using mkvmerge to create a single file without pieces from the cutlist" >> $FALLBACKDIR/ScriptsLastRun.log

mkvmerge --split parts:$keepsecond -o "$FALLBACKDIR/$TVTITLE-$2.mkv" "$MYTHRECORDINGDIR/$7" >> $FALLBACKDIR/ScriptsLastRun.log

#Script assumes if both season and episode is 0 then it must be a movie. Not the best but it works for me.
if [ "$SeasonEpisode" = "s0e0" ]
then

	echo "Attempting to send file to the NAS now" >> $FALLBACKDIR/ScriptsLastRun.log
	mkdir -p "$MOVIEOUTDIR/$TVTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
	mv -u "$FALLBACKDIR/$TVTITLE-$2.mkv" "$FALLBACKDIR/$TVTITLE.mkv" >> $FALLBACKDIR/ScriptsLastRun.log
	mv -u "$FALLBACKDIR/$TVTITLE.mkv" "$MOVIEOUTDIR/$TVTITLE/$TVTITLE.mkv" >> $FALLBACKDIR/ScriptsLastRun.log
else
	echo "Creating folder now:" >> $FALLBACKDIR/ScriptsLastRun.log
	mkdir -p "$TVOUTDIR/$TVTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mkv" "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mkv" #mv -ud in case move fails. 
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mkv" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mkv"
	else
		echo "Running Handbrake on mkv input file now as this is not a movie." >> $FALLBACKDIR/ScriptsLastRun.log
		/usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mkv" -o "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4 >> $FALLBACKDIR/ScriptsLastRun.log
		echo "Attempting to send file to the NAS now" >> $FALLBACKDIR/ScriptsLastRun.log
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode.m4v" "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v"
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v"
		rm "$FALLBACKDIR/$TVTITLE-$2.mkv" #This file is not named right and is before handbrake ran.
	fi
fi

#cleanup the 0 byte mpg file which is created even thou mythtranscode failed.
echo "Removing the temporary mpg file now:" >> $FALLBACKDIR/ScriptsLastRun.log
rm "$FALLBACKDIR/$TVTITLE-$2.mpg" >> $FALLBACKDIR/ScriptsLastRun.log
echo "Script ran to the very end." >> $FALLBACKDIR/ScriptsLastRun.log

exit 0

fi
######## End of MP4 code ############


#################### Code that processes mp2 files ###################

echo "Running Handbrake" >> $FALLBACKDIR/ScriptsLastRun.log


#Need a better way as the following assumes that it is a Movie if the season and episode is both 0
#Current solution is to run different script for TV Shows but left ability in script.

if [ "$SeasonEpisode" = "s0e0" ]
then
	echo "Creating folder now:" >> $FALLBACKDIR/ScriptsLastRun.log
	mkdir -p "$MOVIEOUTDIR/$TVTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
	if [ $MovieQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mpg" "$FALLBACKDIR/$TVTITLE.mpg" >> $FALLBACKDIR/ScriptsLastRun.log
		mv -u "$FALLBACKDIR/$TVTITLE.mpg" "$MOVIEOUTDIR/$TVTITLE/$TVTITLE.mpg" >> $FALLBACKDIR/ScriptsLastRun.log
	else
	/usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mpg" -o "$FALLBACKDIR/$TVTITLE-$2.m4v" -e x264 -q $MovieQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4 >> $FALLBACKDIR/ScriptsLastRun.log
        echo "Attempting to send file to the NAS now" >> $FALLBACKDIR/ScriptsLastRun.log
	mv -u "$FALLBACKDIR/$TVTITLE-$2.m4v" "$FALLBACKDIR/$TVTITLE.m4v" >> $FALLBACKDIR/ScriptsLastRun.log
	mv -u "$FALLBACKDIR/$TVTITLE.m4v" "$MOVIEOUTDIR/$TVTITLE/$TVTITLE.m4v" >> $FALLBACKDIR/ScriptsLastRun.log
	fi
else
	echo "Creating folder now:" >> $FALLBACKDIR/ScriptsLastRun.log
	mkdir -p "$TVOUTDIR/$TVTITLE" >> $FALLBACKDIR/ScriptsLastRun.log
	if [ $TVShowQuality = 0 ]
	then
		mv -u "$FALLBACKDIR/$TVTITLE-$2.mpg" "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mpg"
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mpg" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.mpg"
	else
		/usr/bin/HandBrakeCLI -i "$FALLBACKDIR/$TVTITLE-$2.mpg" -o "$FALLBACKDIR/$TVTITLE-$2.m4v" -e x264 -q $TVShowQuality -O -r 30 --pfr -x ref=6:bframes=5:vbv-maxrate=62000:vbv-bufsize=62000 -X 720 --decomb --loose-anamorphic --modulus 2 --x264-tune film --x264-preset medium --h264-profile high --h264-level 4.1 -a 1,1 -E copy:ac3,faac -B auto,160 -R auto,auto -6 auto,dpl2 --audio-copy-mask aac,ac3,dtshd,dts,mp3 --audio-fallback ffac3 -f mp4 >> $FALLBACKDIR/ScriptsLastRun.log
        	echo "Attempting to send file to the NAS now" >> $FALLBACKDIR/ScriptsLastRun.log
		mv -u "$FALLBACKDIR/$TVTITLE-$2.m4v" "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v" >> $FALLBACKDIR/ScriptsLastRun.log
		mv -u "$FALLBACKDIR/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v" "$TVOUTDIR/$TVTITLE/$TVTITLE - $SeasonEpisode - $TVSUBTITLE.m4v" >> $FALLBACKDIR/ScriptsLastRun.log
	fi
fi

echo "Cleaning up tempory files:" >> $FALLBACKDIR/ScriptsLastRun.log
rm "$FALLBACKDIR/$TVTITLE-$2.mpg" >> $FALLBACKDIR/ScriptsLastRun.log
rm "$FALLBACKDIR/$TVTITLE-$2.mpg.map" >> $FALLBACKDIR/ScriptsLastRun.log
echo "Script ran to the very end." >> $FALLBACKDIR/ScriptsLastRun.log
exit 0
Last edited by skinah on Fri Dec 02, 2016 8:26 am, edited 12 times in total.
Kwisher
Junior
Posts: 58
Joined: Fri Sep 25, 2015 3:02 pm
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by Kwisher »

Great work and thanks for sharing ;)
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

I got the sendTVtonas.sh setup as you documented in the script but when it runs I get an error. Not sure if there are other logs, but the mythbackend log had only the following:

Feb 9 15:56:36 mythbuntu mythbackend: mythbackend[3024]: I UserJob_545 jobqueue.cpp:2409 (DoUserJobThread) JobQueue: Started send TV to NAS for "Hogan's Heroes":"Information Please" recorded from channel 1502 at 2016-02-09T03:30:00Z
Feb 9 15:56:36 mythbuntu mythbackend: mythbackend[3024]: E UserJob_545 jobqueue.cpp:2444 (DoUserJobThread) JobQueue: User Job '/home/jim/sendTVtonas.sh "1502" "20160209033000" "Hogan's Heroes" "Information Please" "2" "15"' failed.

not sure where to start looking. I know that handbrake is working because I can run it from the gui app and convert mpg to m4v.

Any advice?
skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Re: Script for sending Mythtv recordings to a NAS

Post by skinah »

Did you edit the line in the script:
FALLBACKDIR="/home/matt/Videos"
As this line must exist on your install and the user 'mythtv' must have write permissions to write to the location. In this example my user name is 'matt' and by default the mythtv user can not write to this location.

Another useful debug idea is to comment out the last two lines that start with rm that delete and cleanup the files in the script. Add a hash '#'to disable the line. Then browse to the location and tell me if any files are showing up with Hogan's Heroes in the filename, if so tell me the file extension. This will tell me how far the script got before the error occurred.

When you run the script watch the HDD light, does it go solid and work hard for a minute? This will be the part of the script that removes the ads and creates a single MP2 file, then the HDD light will work for ages with minor blinking as the handbrake does its work and slowly writes the MP4 file to your HDD.

My guess will be it is write permissions for the fallback folder or the permissions to the following locations:
MOVIEOUTDIR="/mnt/nas/Movies"
TVOUTDIR="/mnt/nas/TV Shows"

If you run the script manually from a terminal, then it will be you as the user needing write permissions, but when it runs from the mythtvbackend, it will need permission for the mythtv user.

As every line in the script is a terminal command you can try each line manually and see what causes an issue.

Let me know how you go as I will update the script or instructions to assist further people.
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

FALLBACKDIR write permission for mythtv was one of the issues. the other issue was a typo by me in the command. I think it's working now. I'll test some more tonight
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

Thanks for the script and the help to get it working. My NAS is one I built with NAS4Free. I had to go into N4F and setup a user 'mythtv' and a group 'mythtv'. I used the same UID and GID but I don't know if that was required or not. I gave the user a password and put a line in /etc/fstab so it would be mounted so Mythtv could write the files there and create the directories:

Code: Select all

//192.168.0.250/media /media/media cifs username=mythtv,password='mypassword',workgroup=WORKGROUP 0 0
skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Re: Script for sending Mythtv recordings to a NAS

Post by skinah »

No problems, glad other people are finding the script meets their needs as it took ages to create the script as I experimented with handbrake to find a compromise of size and quality that still had surround sound.

You don't need to use mythtv user for accessing the NAS, you can create any user and then simply give the mythtv user write permissions. There is also a way to remove the password from the fstab which is considered more secure, google it if you care.

If you have a iphone/ipad check out the ''infuse'' app, you can then take the recordings with you. Also media companion is a great windows/mac/Linux program for fetching better names that include a year so a movie that has been released multiple times does not show up as the wrong one in Kodi. Conan the Barbarian for example has the same name and released twice.
skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Re: Script for sending Mythtv recordings to a NAS

Post by skinah »

I have updated the movie script to now support MP4 h264 recordings and to also improve the logging so you can read a log file and look for errors. It is not tested extensively with h264 yet so it it works for you please post to let me know.

Post edited so that the latest code is always in the first post of this thread.
Last edited by skinah on Fri Mar 25, 2016 12:44 am, edited 2 times in total.
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

With your original script, you had one for TV and one for Movies. I only used the TV one because everything I record comes from OTA TV, even if it's a movie.

So is the new script only for Movies or is it also for TV? I guess what I'm asking is, what is the difference between when you use one script or the other?
skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Re: Script for sending Mythtv recordings to a NAS

Post by skinah »

The Movie script makes an assumption that if a recording has a empty season and episode number then it must be a movie. It then uses better compression for better quality, names it in a movie format and places it in a kodi compatible location for movies. The same script can also do TV shows that are in a series like 'The Office' as it should have a season and episode number. It then uses higher compression, names it with the season and episode and places it in a folder for Kodi.

The problem arises when you record something like 'The world cup' which is not a movie and also not a TV series, this is when I use the TV script that you have. Otherwise the recording would show up in KODI under the movie section. If you wanted to trade file size for better quality you could use the alternative handbrake setting to learn how to tweak it for yourself.

I am yet to work out a way to use the same script to process everything in one. I'll update the TV script after testing this one for a while to ensure the newer h264 lossless cuts are working. If you dont have h264 recordings then the only new thing is better log file creation to help fault find.
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

I've been using this script to remove commercials using my cutlist based on the commercial flags and my manual edits and move the programs to my NAS. All successfully. I've been testing everything with 0.28 mythtv and I've run into an issue with sendTVtonas.sh.

I think it has to do with the fact that the recorded files have a .ts extension now in 0.28. After the script strips the commercials without converting it to .m4v it seems to remove it from the directory it was originally stored. The database seems to think the file is still there, but can't find it. My Videos directory (FALLBACKDIR) has the .mpg, m4v and .map files while it's doing the transcoding. After the transcoding it complete the Videos directory is empty and the NAS has my completed .m4v file. So other than the original .ts file being deleted, it works.

I have not dug into the script yet myself, but you may have an idea.
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

jfabernathy wrote: I think it has to do with the fact that the recorded files have a .ts extension now in 0.28. After the script strips the commercials without converting it to .m4v it seems to remove it from the directory it was originally stored.
during the first phase where the commercials are removed by a lossless transcoding the .ts files remains, but once that is complete and the handbrake starts the .ts file disappears without the database being updated. I'm not sure if the lossless transcode deletes the .ts or handbrake. But the handbrake command does work and the .m4v file does make it to the NAS.

On further testing manually running the mythtranscode command I find it deletes the original .ts file without updating the database. In version 0.27 the original file remained unmodified. I think this is a bug and started a separate thread in the Linux sub-forum.
skinah
Junior
Posts: 53
Joined: Sat Oct 17, 2015 11:56 pm

Re: Script for sending Mythtv recordings to a NAS

Post by skinah »

The script 'should not' touch your database nor touch any of your recording files. It should only make a compressed copy and move this copy not the original file to your NAS. If you find it is doing one of those two things then yes I would agree it is most likely a bug with the mythtranscode command which the script uses to remove commercials and you can test any of the lines in the script in a terminal to prove which line creates an issue. If you need to know the channel ID and timestamp of a recording for testing the LOG file for this script should list the last file you ran it on so you can grab it from there. If reporting a bug you would be best not to mention your using a script, first work out if it happens with the same command the script uses in a terminal and then report the bug on the command ie mythtranscode. Your more likely to get quicker help doing that. I'm still using 0.27 for the moment as my setup is working well and I never touch or apt-get update a working myth setup.
User avatar
jfabernathy
Senior
Posts: 577
Joined: Wed Feb 18, 2015 2:37 pm
Location: Raleigh, NC
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by jfabernathy »

I submitted a bug report on the mythtranscode command with the -o option in 0.28. the 0.28 version does indeed delete the original file and remove the cutlist. 0.27 does not and that's what I like your script. I'm staying on 0.27 until this is fixed in 0.28.
Kwisher
Junior
Posts: 58
Joined: Fri Sep 25, 2015 3:02 pm
United States of America

Re: Script for sending Mythtv recordings to a NAS

Post by Kwisher »

jfabernathy wrote:I submitted a bug report on the mythtranscode command with the -o option in 0.28. the 0.28 version does indeed delete the original file and remove the cutlist. 0.27 does not and that's what I like your script. I'm staying on 0.27 until this is fixed in 0.28.
I am finding the same issue since moving to .28. My transcode script worked perfectly in .27 also.
See this topic: viewtopic.php?f=36&t=1037
Post Reply