Default Storage Group directory assignment

williamatwood
Junior
Posts: 63
Joined: Sun Feb 12, 2017 8:08 pm
Canada

Default Storage Group directory assignment

Post by williamatwood »

In a previous version of MythTV (probably 0.27), several directories were defined in /var/lib/mythtv, and they were all set to have owner:group mythtv:mythtv and permissions 775. As a result, when the backend is run as user "mythtv", it was possible for the backend to write to these directories. The default assignment for each of the "Storage Groups" was one of these directories. As a result, it was possible to watch TV, without any explicit action on the part of the user.

In the current version (0.28.1), although the directories are created in /var/lib/mythtv, the defaults for the storage groups are empty. Therefore, before _anything_ will work, the user must assign real directories to (at least) the "Default" storage group, and this directory must be writable by user "mythtv".
My environment:
OS: Description: Ubuntu 16.04.2 LTS
Kernel: 4.4.0-64-generic
MythTV: Installed: 2:0.28.1+fixes.20170214.299dbae-0ubuntu0mythbuntu3

I also observe the following differences between 0.27 and 0.28:
Storage Groups "Photographs", "Music", and "MusicArt" have been added in 0.28, but no directory is created in /var/lib/mythtv to correspond to these.
Storage Group "Recordings" probably existed (my records are incomplete) in 0.27. It no longer exists in 0.28, but the directory still does in /var/lib/mythtv.
Directory /var/lib/mythtv/streaming exists in 0.28, but there is no corresponding Storage Group.

If it has not already been done in a later update to 0.28.1, I suggest that you revert to setting a default directory for each displayed Storage Group. At least, with the directory and its permissions visible, the user will be able to create useful new ones if he/she wishes. I also suggest that the initial list of Storage Groups (including the "Default" group) have a corresponding directory in /var/lib/mythtv.
User avatar
pgbennett
Developer
Posts: 503
Joined: Mon Apr 27, 2015 5:41 pm
United States of America

Re: Default Storage Group directory assignment

Post by pgbennett »

Your suggestion sounds like a good idea, but there could be complications.

MythTV is supported on many platforms. Some users build from source and others install from a package. I think it may be problematic if the database was initialized with directories such as /var/mythtv/lib/default. In the case of a non-linux build those directories may be impractical or invalid. In the case of a user who built his own system from source the directories may not exist.

Currently if those directories are created, it is done by packaging teams. That would have to be moved to the configure and install process so that all systems are consistent.
williamatwood
Junior
Posts: 63
Joined: Sun Feb 12, 2017 8:08 pm
Canada

Re: Default Storage Group directory assignment

Post by williamatwood »

I realize that this is a packaging issue, rather than a true mythytv issue. However, I could see no way of attracting the attention of the packagers, so I chose this sub-forum to get the discussion started. If I should be using a different sub-forum, please tell me which one.

I _did_ use a package, targeted at Ubuntu systems, and installed with "sudo apt-get-install mythtv". I guess my point is that such a package should be built so that, after installation, the result has some hope of working. 0.27 packaging satisfied that requirement. 0.28 packaging (apparently) does not.
kd6ybr
Junior
Posts: 15
Joined: Sun Jan 17, 2016 9:10 pm
Great Britain

Re: Default Storage Group directory assignment

Post by kd6ybr »

I would have to agree with williamatwood that the decision to remove sensible default storage directories​ from 0.28 is a step backwards for mythtv. Surely the priority should be to make mythtv as simple as possible to install on a Linux system using prebuilt packages? And if that means someone has to do a bit of extra work trying to install mythtv from source on a non-linux machine then so be it.

Since 14.04 it has gotten much harder to get a working system easily installed. HDMI doesn't work on 14.04 and 0.28/16.04 has the storage directory problem as well as the HDMI issue. Mythbuntu is being dropped at the same time that xubuntu, the recommended distribution for installing mythtv on, hasn't supported HDMI since 13.10.

I do appreciate all the hard work that the developers do but not many other people will if they can't easily get a system that works reliably.
Gigabyte Brix GB-BXBT-2807 4gb 120gb ssd
Sony PlayTV DVB-T USB tuner
Mythbuntu 14.04 0.27 FE&BE
User avatar
pgbennett
Developer
Posts: 503
Joined: Mon Apr 27, 2015 5:41 pm
United States of America

Re: Default Storage Group directory assignment

Post by pgbennett »

Why do you think HDMI does not work? I have 2 TV's connected using HDMI with xubuntu 16.04. Before that I had them on ubuntu 12.04. I get 1080 resolution plus surround sound audio, all through HDMI. I never had a problem with HDMI. Let us know what problem you have with it.

The storage directory issue could be subject to debate. The default locations that were provided were likely in the user's root file system, and somebody using that default setup could be setting themselves up for problems when their root file system fills up. Serious users of MythTV need to set up one or more file systems dedicated to storage of recordings.
perkins1724
Junior
Posts: 33
Joined: Tue Nov 18, 2014 7:44 am
Australia

Re: Default Storage Group directory assignment

Post by perkins1724 »

HDMI doesn't work on 14.04
MythTV works fine with HDMI in both fixes/0.27 and fixes/0.28 (and many other branches). If HDMI is not working on Ubuntu 14.04 or 16.04 (or Xubuntu) then that is definitely an issue well outside the control of the Ubuntu mythtv packagers or the mythtv devs, it would be a core Ubuntu (Xubuntu) issue.
the decision to remove sensible default storage directories​ from 0.28 is a step backwards
It may be a controversial viewpoint, but I don't think the mythtv dev's should be attempting to set default storage directories. Not only because given the wide range of distro's that any choice will be just as broken as no choice. But it is worth pointing out that the consequences of a full OS disk can be severe so whether people like it or not being forced to think about the storage setup - at the start and before dropping huge GB of recordings onto disk - is a good thing (IMO).

Now at the packaging level, sure, if that is the direction they want to take. At least the packagers have a high level of certainty as to file system configuration so it may make sense at the packager level - depending on distro.

With reference to Ubuntu and this thread, I'm not involved in the packaging (or even a *buntu user) so I can't comment as to why they may have dropped /var/lib/mythtv from the default storage group. It may be accidental or it may be deliberate. But I do know that that before Mythbuntu was dropped the maintainers put out a call for assistance due to a lack of resources to keep the project going. There was a limited positive response but I think there is still essentially only two main people to keep the mythtv ppa working and resourcing is a major problem. It is worth noting that even Mythbuntu 16.04 was nearly dropped so it has been a problem for a long time.

I can think of at least two other Ubuntu packaging concerns (handling / passing of the randomised password to a desktop user running mythfrontend, and mysql 5.7 compatibility requiring adding 'sql_mode=NO_ENGINE_SUBSTITUTION' to /etc/mysql/conf.d/mythtv.cnf) that have been open for a period of time and have detracted from the UX. I suspect that resourcing plays a major part in at least part of these.

I believe that Ubuntu is an important entry point for many starting out in MythTV and therefore having it work out-of-the box will give the best first impression. But if there is currently insufficient resources to achieve this then the question is how to boost the resourcing not what should the defaults look like.
kd6ybr
Junior
Posts: 15
Joined: Sun Jan 17, 2016 9:10 pm
Great Britain

Re: Default Storage Group directory assignment

Post by kd6ybr »

The HDMI issue is well documented in this thread as well as a work around for 14.04.

viewtopic.php?f=29&t=1165&p=6105#p6105

My concern for mythtv is that with no current easy entry point for newbies the project gets into a spiral of lack of developers because no one is using it.

My recommendation is for newbies to install mythbuntu 14.04 simply because it is the easiest option to get a working mythtv system quickly.
Gigabyte Brix GB-BXBT-2807 4gb 120gb ssd
Sony PlayTV DVB-T USB tuner
Mythbuntu 14.04 0.27 FE&BE
perkins1724
Junior
Posts: 33
Joined: Tue Nov 18, 2014 7:44 am
Australia

Re: Default Storage Group directory assignment

Post by perkins1724 »

The HDMI issue is well documented in this thread as well as a work around for 14.04.

viewtopic.php?f=29&t=1165&p=6105#p6105
Thankyou for linking the details.

The bug linked in that thread indicates that it has been patched in xfce 4.12.1 and it appears current version in Ubuntu 16.04.01 is xfce 4.12.2. I don't run Ubuntu so can't confirm but the comments in the bug report seem quite positive so it seems it is now (at long last) resolved.

Otherwise the bug report indicates a workaround for Xubuntu 16.04.01 is to:

Code: Select all

dpkg -i xfce4-settings_4.12.0-3ubuntu1_amd64.deb && apt-get install -f
again I can't verify if it works but comments are favourable.

My recollection of the problem and workaround (quite some time ago, as you quite rightly point out it has been around a long long time) was to simply just disable xfsettingsd if you didn't need the xfsettingsd functionality. Either add killall xfsettingsd to /etc/mythtv/session-settings or disable xfsettingsd autostart. It was certainly not perfect I agree but hopefully functional for some.
My concern for mythtv is that with no current easy entry point for newbies the project gets into a spiral of lack of developers because no one is using it
I agree. These exact types of issues can very, very quickly turn new users away. And when it comes to Ubuntu 16.04 there do appear to be a couple of common hurdles that people are finding difficult to get past which is unfortunate.
User avatar
pgbennett
Developer
Posts: 503
Joined: Mon Apr 27, 2015 5:41 pm
United States of America

Re: Default Storage Group directory assignment

Post by pgbennett »

kd6ybr wrote:The HDMI issue is well documented in this thread as well as a work around for 14.04.

viewtopic.php?f=29&t=1165&p=6105#p6105

My concern for mythtv is that with no current easy entry point for newbies the project gets into a spiral of lack of developers because no one is using it.

My recommendation is for newbies to install mythbuntu 14.04 simply because it is the easiest option to get a working mythtv system quickly.
Personally, I ran Ubuntu 12.04 for 4 years and later upgraded to Xubuntu 16.04. I never saw this problem, never had any issue with HDMI. So I suppose I was lucky. It looks like that was a only problem with version 14.10 of Ubuntu. Also looking at that thread it seems it was only a problem if you switch your TV to another device then back to Ubuntu. So it is a long way from the statement "HDMI doesn't work" and "hasn't supported HDMI since 13.10".
kd6ybr
Junior
Posts: 15
Joined: Sun Jan 17, 2016 9:10 pm
Great Britain

Re: Default Storage Group directory assignment

Post by kd6ybr »

pgbennett wrote:Also looking at that thread it seems it was only a problem if you switch your TV to another device then back to Ubuntu. So it is a long way from the statement "HDMI doesn't work" and "hasn't supported HDMI since 13.10".
I don't think I am in the minority of users who have more than one device connected to their TV. :shock: :D

Yes, you may think I am being over dramatic in saying "HDMI doesn't work" but when my family switch to mythtv and get a black screen then as far as they are concerned HDMI doesn't work. :(

It does look like a fix is in the works for the HDMI issue but wasn't in the repos last week when I ran an update on 16.04 before rolling back to 14.04.

Getting back on topic a decision was made in 0.28 to drop default directory assignments and force users to set them up. While this may make sense for people setting up complex mythtv systems an extra hurdle has been added to those wanting to try out mythtv. I would suggest that my system (see signature) is probably a good representation of a system someone would use to try out mythtv. It should be possible to install a system like this in an hour or 2 and have a reliable working system following a tutorial. 16.04 doesn't meet this requirement, 14.04 is close with a few tweaks.

So with 0.28 a decision was made to add a hurdle in front of new users by removing default storage directories. I feel that if you have to choose where to put the hurdles, it would make more sense to put the hurdles in front of the people who are prepared to jump them. Don't put hurdles in front of new users, try and remove as many hurdles from new users as possible. If that means more hurdles for advanced users, people trying to install mythtv on non-linux machines, people with multiple FE and BE, then so be it. Those people are willing to jump a few hurdles. Make the experience as hurdle free for new users and you will get more people using and offereing to further develop mythtv in the future.

A reliable way to install a combined FE&BE on a single hard drive machine is desperately required. :) :)

Greg
Gigabyte Brix GB-BXBT-2807 4gb 120gb ssd
Sony PlayTV DVB-T USB tuner
Mythbuntu 14.04 0.27 FE&BE
User avatar
bill6502
Developer
Posts: 2307
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: Default Storage Group directory assignment

Post by bill6502 »

There's no MythTV code that adds /var/lib/mythtv/recordings. As williamatwood
pointed out earlier, this appeared to be a packaging issue. But we don't have any logs in this
thread, so it's not possible to get the root cause of the failure.

I did a git checkout fixes/0.28 and don't see any references to /var/lib/mythtv. Same
for 0.27. However, if no Storage Directories were added to the Default Storage
Group, then the recordings would go to the hardcoded /mnt/store directory. Users
are alerted to this in the backend log like this:

Code: Select all

...Unable to find any Storage Group Directories.  Using hardcoded default value of '/mnt/store'
Which is fine if a user mounts a file system on /mnt and it has a directory
named store on it. But if that directory was created in /mnt/store, the root
file system would fill up quickly. [comment edited 5/1/2017]

This is a tough nut to crack. I agree with Greg's point about making the project
friendlier, but users should be making informed decisions about their storage.

The existing GUI has a good (my opinion) section dedicated to Storage Groups,
and I even tried to delete the Default one - and it can't be done. If Storage
Directories are added and the user doesn't have write permission, then a
reasonable warning message prints as mythtv-setup is exited.

Side note, Mythbuntu (as a distribution) is going away: http://www.mythbuntu.org/home/news
kd6ybr
Junior
Posts: 15
Joined: Sun Jan 17, 2016 9:10 pm
Great Britain

Re: Default Storage Group directory assignment

Post by kd6ybr »

Going slightly off topic again, but maybe a mythtv snap package that could be installed on ubuntu-server could be a way forward to get new users. This approach has worked well for the nextcloud team, their snap package has limitations but does install a working nextcloud system with minimal work.
Gigabyte Brix GB-BXBT-2807 4gb 120gb ssd
Sony PlayTV DVB-T USB tuner
Mythbuntu 14.04 0.27 FE&BE
dude
Newcomer
Posts: 5
Joined: Sun Oct 06, 2019 10:04 pm
Great Britain

Re: Default Storage Group directory assignment

Post by dude »

The existing GUI has a good (my opinion) section dedicated to Storage Groups,
and I even tried to delete the Default one - and it can't be done. If Storage
Directories are added and the user doesn't have write permission, then a
reasonable warning message prints as mythtv-setup is exited.

Hi Bill,

Thanks for your tireless efforts on this and any other programs you have designed. I really appreciate it. I have been using various Linux distros for a few years, and I really appreciate all of the hard work and dedication that goes into everything. Manjaro is my current favorite.

How exactly do I change the file permissions on the "Storage Groups"? <<<< this is the trillion dollar question...

I understand that you say that "upon exiting mythtv-setup" a warning message is displayed, however, there is no guidance given as to how to rectify the condition/problem. This is why there are 99 trillion hits on Google about "mythtv scanning channels, or parsing errors, or error opening buffer file" I believe if the instructions from professionals such as yourself were more descriptive, this would alleviate half of the inquiries/questions about common tasks.

In my opinion, this is the problem with Linux distros in general; the standard reply is: "it's in the WIKI", however, the wiki is not always cut and dry for a person that is not experienced with CLI commands such as yourself. I personally think Wikis should be required to be labeled by a year or version, because so many things change with different versions or years...etc.

Like I said earlier, I appreciate all of the hard work and dedication that has gone into all of the development of these apps/programs, but I think a more detailed explanation of the specific CLI commands would go a long way in reducing the amount of repetitive posts/threads about the recurring questions/problems. This, in turn, would reduce the amount of replies for experienced people such as yourself.

Once again, thanks for all of your hard work on these and any future apps/programs.

The open-source community really appreciates it.
MikeB2013
Senior
Posts: 519
Joined: Mon Jul 25, 2016 4:16 pm
Great Britain

Re: Default Storage Group directory assignment

Post by MikeB2013 »

MythTV Storage Directories Permissions

For Ubuntu/Debian based systems mythbackend runs under user mythtv.
The storage directories need to be owned by mythtv and have appropriate read/write permissions.

On a MythTV install from mythbuntu ppa 30 the following default directories are created under /var/lib/mythtv/

Code: Select all

mike@mike-GL62-7QF:~$ ls -lha /var/lib/mythtv
total 56K
drwxr-xr-x 12 root   root   4.0K Jun 28  2018 .
drwxr-xr-x 89 root   root   4.0K Jul 21 10:39 ..
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 banners
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 coverart
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 db_backups
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 fanart
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 livetv
drwxrwsr-x  2 mythtv mythtv  12K Mar 12  2019 recordings
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 screenshots
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 streaming
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 trailers
drwxrwsr-x  2 mythtv mythtv 4.0K Apr 26  2018 videos
The commands to set ownership and permissions are :

Code: Select all

sudo chown -R mythtv:mythtv /var/lib/mythtv/
sudo chmod -R  2775 /var/lib/mythtv/
The -R applies the command to all subdirectories under /var/lib/mythtv/

Mike
dude
Newcomer
Posts: 5
Joined: Sun Oct 06, 2019 10:04 pm
Great Britain

Re: Default Storage Group directory assignment

Post by dude »

Thanks MikeB2013

I followed your detailed advice and Live TV is now working for me. Thanks for your detailed explanation/response.

None of my HD channels are working, but I think I also have a dish alignment problem. I have also noticed poor reception on my Enigma2 receiver on some channels on 28.2E.

This is the first time I have used MythTV, the options/interface are much better than TVHeadend, however, the initial setup is not so easy.

Once again, thanks.
Post Reply