Cleaning up livetv recordings

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Cleaning up livetv recordings

Post by gpw928 »

I have been using MythTV for a while, and the livetv directory has grown to 85GB.

Dr Google tells me that the stuff from 2011 should probably have auto-expired by now, but it has not:

Code: Select all

[umtvs1.546] $ cd /var/lib/mythtv/livetv
[umtvs1.547] $ du -h -s .
85G
[umtvs1.548] $ ls -al | wc -l
261
[umtvs1.549] $ ls -lart | tail -5
-rw-r--r--  1 mythtv mythtv  131410872 Feb 14 21:31 10083_20230214102412.ts
-rw-rw-rw-  1 mythtv mythtv      85721 Feb 14 21:31 10083_20230214102412.ts.png
-rw-r--r--  1 mythtv mythtv  178342816 Feb 14 21:37 10008_20230214103145.ts
-rw-r--r--  1 mythtv mythtv  449264916 Feb 14 22:35 10008_20230214111835.ts
drwxrwsr-x  2 mythtv mythtv        260 Feb 15 12:48 .
[umtvs1.550] $ ls -lat | tail -5
-rw-r--r--  1 mythtv mythtv   29363344 Oct  2  2011 1024_20111002134856.mpg
-rw-rw-rw-  1 mythtv mythtv     106466 Oct  2  2011 1024_20111002134856.mpg.png
-rw-r--r--  1 mythtv mythtv          0 Aug 30  2011 1003_20110830203127.mpg
-rw-r--r--  1 mythtv mythtv    3879944 Aug 30  2011 1003_20110830203044.mpg
-rw-r--r--  1 mythtv mythtv          0 Aug 30  2011 1003_20110830203043.mpg
Presumably, the database is out of sync with reality here.

Is there a safe way to delete all the livetv recordings (there's nothing in there that I want)?
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

I have been digging into the database, and think I have found what's needed to remove all the files in the livetv storage group:

Code: Select all

sudo systemctl stop mythtv-backend
/usr/share/mythtv/mythconverg_backup.pl
sudo mysql -u root -p mysql # use root's password
use mythconverg;
delete * from recordedfile where storagegroup = 'livetv';
flush privileges;
exit;
cd /var/lib/mythtv/livetv
rm -rf *
sudo systemctl start mythtv-backend
I'd appreciate confirmation of this approach.
Last edited by gpw928 on Wed Feb 15, 2023 11:27 am, edited 1 time in total.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Cleaning up livetv recordings

Post by paulh »

Could be a permissions problem I wonder if those files need to allow the mythtv user to both read and write them? That should them allow the backend to cleanly remove them.
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

The /var/lib/mythtv/livetv directory is readable and writable by mythtv:mythtv. It's also setgid, so new files get the mythtv group.

There's a 100% consistent pattern that recordings (.ts files) are mode 644 and thumbnails (.ts.png files) are mode 666.

Because of the permissions, anyone who is a member of the mythtv group has write permission to the directory /var/lib/mythtv/livetv, and can thus delete any file in that directory. So I don't think it's a permission problem.

I'm currently working on extracting a list of files using "select basename from recordedfile where storagegroup = 'livetv'". I'll compare that to what's in the directory. But I already know that there's 135 rows in the SQL table, and 239 files in the directory. So I'm fairly sure that the database needs to be corrected.

Edit: there are:
  • 124 files in the directory /var/lib/mythtv/livetv and also in the recorded file table;
    11 files unique to the recorded file table (not in the directory /var/lib/mythtv/livetv); and
    115 files unique to the directory /var/lib/mythtv/livetv (not in the recorded file table).
Last edited by gpw928 on Wed Feb 15, 2023 12:03 pm, edited 1 time in total.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Cleaning up livetv recordings

Post by paulh »

I'm no linux expert but it looks like only root has read/write permissions on the recording files so if you run the backend as user mythtv it can't slow delete those files. At least that is my theory :roll:
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

To delete a file you need write permission on the directory in which it resides.

Anyone who is a member of the mythtv group has write permission to the directory /var/lib/mythtv/livetv, and can thus delete any file in that directory.
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

I am quite confident that the auto-deletions are working for recent livetv recordings. If they were not, then there would be an awful lot more than 239 files in the /var/lib/mythtv/livetv directory.

However, the directory /var/lib/mythtv/livetv has existed for more than 12 years and for many iterations of MythTV versions, and database upgrades.

I'm thinking that gremlins in auto-deletions at some time(s) in the past have got the SQL table out of sync with the contents of the directory.

So all I really want to do is delete 100% of live recordings.

I think that the code in my second post will do it, and I'd appreciate critique from somebody familiar with the workings of the database.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Cleaning up livetv recordings

Post by paulh »

You can filter recordings by LiveTV in watch recordings. Press menu and select Change Group Filter. Find the Live TV option. If it's missing then you have no LiveTV recordings. You can choose to delete them one by one. Then in theory anything that is left in the LiveTV storage group has become orphaned and can be deleted.

EDIT
And for clarity I don't know if LiveTV recordings are slow deleted or not I would assume they are if it is enabled for recordings so they are slowly truncated and then deleted so if you have a lot of big LiveTV recordings it's would be best to wait a while before assuming what is left are orphaned.
gedakc
Junior
Posts: 96
Joined: Fri Jul 18, 2014 1:28 am
Canada

Re: Cleaning up livetv recordings

Post by gedakc »

Another thing to check is if you have encountered the following problem with removing recordings which are pending deletion:

Massive disk space wastage
MythTV Wiki: User Manual:Periodic Maintenance - Removing recordings which are pending deletion

I occasionally have this problem with my Mythbuntu 14.04 (MythTV 0.27) PVRs. It seems to be related to if the PVR is configured to use mythwelcome which permits automatic shutdown and later power up to record programs.

The commands to check for the problem are included in the above links.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Cleaning up livetv recordings

Post by paulh »

gedakc you could try a similar thing to actually delete recordings. On the Watch Recording screen you can filter the recording list showing Deleted programs. You can try to select each one and press 'D' to actually delete it. If you have a lot of deleted recordings you can create a playlist from the Deleted group and choose to Delete all of them.

Again if slow delete is on each recording will be slowly truncated in size before being finally deleted so don't expect them to instantly disappear. It can be interesting to watch the used and free sizes changing on the Watch Recording screen if your theme shows it.

You can also force a delete of deleted and auto expired recordings on the AutoExpire List tab on the System Status page.
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

gedakc wrote:
Wed Feb 15, 2023 4:59 pm
Another thing to check is if you have encountered the following problem with removing recordings which are pending deletion
Thanks for the links. It looks like my problems go a bit further than the livetv group:

Code: Select all

mysql> select deletepending, recgroup, count(*) from recorded group by deletepending, recgroup;
+---------------+----------+----------+
| deletepending | recgroup | count(*) |
+---------------+----------+----------+
|             1 | LiveTV   |       62 |
|             0 | Default  |     2339 |
|             1 | Deleted  |     1748 |
|             0 | LiveTV   |       30 |
|             0 | Deleted  |        7 |
+---------------+----------+----------+
Oops. 1748 deleted recordings. I will follow the Wiki link to fix that, and report the results.
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

paulh wrote:
Wed Feb 15, 2023 12:40 pm
You can filter recordings by LiveTV in watch recordings. Press menu and select Change Group Filter. Find the Live TV option.
I have found the LIveTV recordings from the remote control, and can watch them, but have not been able to figure out how to delete them. Nothing I have tried, including "D" on the keyboard, works.

I'm still keen for somebody familiar with the schema to comment on the proposed method to clean up the livetv directory in my second post.

[I'm pretty sure it's correct, but it may not be complete, in the sense that other parts of the schema may also need surgery.]
gedakc
Junior
Posts: 96
Joined: Fri Jul 18, 2014 1:28 am
Canada

Re: Cleaning up livetv recordings

Post by gedakc »

Thank you paulh for the alternate suggestions for deleting recordings. I do have slow delete enabled and am quite happy to let mythbackend take the time to slowly delete these recordings (usually at least 20 minutes or more).

EDIT:
gpw928 that sure is a lot of recordings with a "1" indicating delete pending. Did you invoke the eliminate deletepending recordings command so that mythbackend can start to perform the deletes? If so you can check on the progress by running the showpendings command.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Cleaning up livetv recordings

Post by paulh »

gpw928 that's strange maybe you have the 'D' key bound to something else or the Global->DELETE key binding is set to something other than 'D'?

You can also press 'M' to show the menu and choose 'Delete' from there.
gpw928
Junior
Posts: 51
Joined: Tue Feb 12, 2019 5:59 am
Location: Rural NSW
Australia

Re: Cleaning up livetv recordings

Post by gpw928 »

paulh wrote:
Wed Feb 15, 2023 8:43 pm
gpw928 that's strange maybe you have the 'D' key bound to something else or the Global->DELETE key binding is set to something other than 'D'?

You can also press 'M' to show the menu and choose 'Delete' from there.
I rarely use the keyboard, and need to refer to the manual for even the most basic keyboard operations. But I have not changed any of the default keyboard bindings -- that I know of.

I have installed a custom keytable for the IR remote.

I got to the LiveTV listings by watching live TV, and then, on the keyboard:
  • press "M" -> Playback Menu
    press "Return" to choose Source (from menu)
    press "Return" to choose Jump to Program (from menu)
    press "Return" to choose Recorded Program (from menu)
    press "Return" to choose Live TV (from menu)
    press "Return" to choose Recorded Program (from menu)
This produces a list of recorded programs that I recognise.

Press the down arrow to highlight a recorded program.
  • press "D" -> nothing happens
    press "M" -> returns immediately to the current broadcast live TV
Post Reply