Whilst migrating from a 0.27 system to version 31 I uncovered several hundred mpg files consuming 662GB of disk which were spurious.
Removing them took the free space on my 2TB disk from 18% to 45% - I did not need to buy that 4TB disk for my new system after all!!
The spurious recording files are not listed by frontend and are not discovered by find_orphans.py. The 0.27 GetRecordedList API shows them with a status 'delete pending' but under 31 they are not shown at all. I have run the optimize_database script regularly.
Now the accepted wisdom is that such files are left behind if you have a system crash shortly after requesting a deletion but the accretion rate of one of these every week together with the very low failure rate of my system suggest another mechanism.
I suspect that this behaviour will not be shown if your backend is reliable and is permanently on; it only shows if you closedown regularly such as with mythwelcome. My hypothesis of how they accumulate is as follows. We will typically record (say) 10 weekly episodes of a series and then 'binge watch' them. We will then delete them all then escape to the mythwelcome screen which closes down after a 30 second delay. That backend closedown whilst the recordings are still being deleted causes the problem.
The fix is revealed here:
viewtopic.php?f=36&t=4735 and in viewtopic.php?f=36&t=4071 (thanks to Roland and PGBennett!!).
You can run this script to check whether you have this problem
showpendings.sh
Code: Select all
#!/bin/bash
# mysql commands to show deletepending recordings
PASS=`grep Password ~/.mythtv/config.xml | sed 's/<\/*Password>//g' | sed 's/ *//g'`
SQLLINE='select deletepending, recgroup, count(*) from recorded group by deletepending, recgroup;'
echo $SQLLINE | mysql --database=mythconverg --user=mythtv --password=${PASS}
You can expected a response like this:
That 423 figure is the number of these spurious recordings. The 'Deleted' line will be absent if there are none.mysql: [Warning] Using a password on the command line interface can be insecure.
deletepending recgroup count(*)
0 Default 643
1 Deleted 432
You can remove the spurious recordings (under 0.27 or 31) with this script run at a time when backend is idle:
Code: Select all
#!/bin/bash
# mysql commands to eliminate deletepending recordings
PASS=`grep Password ~/.mythtv/config.xml | sed 's/<\/*Password>//g' | sed 's/ *//g'`
SQLLINE='select deletepending, recgroup, count(*) from recorded group by deletepending, recgroup;'
REPLY=`echo $SQLLINE | mysql --database=mythconverg --user=mythtv --password=${PASS} | grep Deleted`
[ -z "$REPLY" ] && exit
echo clearing deletepending flags
systemctl stop mythtv-backend.service
SQLLINE="UPDATE recorded SET deletepending = 0 where recgroup = 'Deleted';"
echo $SQLLINE | mysql --database=mythconverg --user=mythtv --password=${PASS}
systemctl start mythtv-backend.service
Then leave backend running to delete the files and their entries in the database - allow about 10 sec/recording. Use showpendings.sh whilst this is going on to check progress.
My solution with my new system which will be 'always on' is a 3am database backup if everything is idle and a monthly optimize. I'll include the fix at that monthly optimize. That does not fix the problem for those systems which are closed down after use.
I'll get round to updating the routine maintenance pages in the wiki, but does the forum think that changes need requesting to major and important mythtv components?
- Including the sql fix during backend initiation should fix this but is that a good solution?
- Should we include it in optimize?
- Should the API change which stopped listing such recording be reversed?
- Is this a bug in find_orphans.py?
Comments appreciated!
Phil