SOLVED: Music storage group

Have a MythTV related problem? Ask for help from other MythTV users here.

Moderator: Forum Moderators

Post Reply
marcus_hall
Newcomer
Posts: 5
Joined: Tue Aug 02, 2016 1:09 pm
United States of America

SOLVED: Music storage group

Post by marcus_hall »

I am running the distributed packages from fedora, and recently upgraded to fedora 24, which brings mythtv 0.28. Everything seemed to go well with the upgrade, with one problem. After the upgrade, the music plugin was not working, complaining that the Music storage group needed to be defined. OK, shouldn't be a big deal. I went into myth-setup on the backend, went to storage groups, and defined the Music storage group to be "/var/lib/mythtv/music/", which is the directory holding the music tree. Tried again (I think I was importing files), and it complained about MusicArt not being defined. OK, define that and created a directory "/var/lib/mythtv/musicart". Empty for now, and I cannot find anything on what the directory layout within it should be.. Currently, I have "albumart.jpg" files in each album in the music tree, but it appears that this will no longer be the standard.. OK, I'll figure that out eventually...

Anyhow, now importing files no longer complains about missing storage groups, but still after claiming to have added new songs to the database, I get messages that the copy into the music tree has failed. Indeed, the files did not make it into the tree, and actually the database was not updated. Running the backend with -v file:debug, I see an attempt to delete the file first, which fails saying it doesn't exist. Seems normal, but then when attempting to start the copy:

2016-08-01 12:17:38.737943 D [8924/9041] ProcessRequest storagegroup.cpp:687 (FindNextDirMostFree) - SG(Music): FindNextDirMostFree: Starting
2016-08-01 12:17:38.737956 E [8924/9041] ProcessRequest storagegroup.cpp:726 (FindNextDirMostFree) - SG(Music): FindNextDirMostFree: Unable to find any directories to use.
2016-08-01 12:17:38.737962 E [8924/9041] ProcessRequest mainserver.cpp:1867 (HandleAnnounce) - MainServer: Unable to determine directory to write to in FileTransfer write command


So, I believe that the backend somehow isn't using the storage group definition, or something. When I
attempt to play a song, I see the backend doing:

2016-08-01 20:44:56.128467 I [13325/13375] ProcessRequest mainserver.cpp:3795 (HandleQueryFindFile) - MainServer: Looking for file 'Stewart, Al/Between The Wars/03-Sampan.flac' on host 'brinton' in group 'Music' (useregex: 0, allowfallback: 1
2016-08-01 20:44:56.128477 I [13325/13375] ProcessRequest mainserver.cpp:3800 (HandleQueryFindFile) - MainServer: Checking local host 'brinton' for file
2016-08-01 20:44:56.128964 D [13325/13375] ProcessRequest storagegroup.cpp:608 (FindFile) - SG(Music): FindFile: Searching for 'Stewart, Al/Between The Wars/03-Sampan.flac'
2016-08-01 20:44:56.128991 E [13325/13375] ProcessRequest storagegroup.cpp:622 (FindFile) - SG(Music): FindFile: Unable to find 'Stewart, Al/Between The Wars/03-Sampan.flac'!
2016-08-01 20:44:56.129339 I [13325/13375] ProcessRequest mainserver.cpp:3946 (HandleQueryFindFile) - MainServer: File was not found

The file in question definately exists under /var/lib/mythtv/music, so again it appears to be something related to storage group definition. I can play videos just fine, with the storage group similarly named /var/lib/mythtv/videos.

Now, both of these directories are NFS mounted as well, so if there's any way to just turn off storage groups and go with NFS, I'm good with that, but mythtv seems to be all into reinventing remote file sharing one more time, but, whatever..

I did notice that during startup, the backend seems to create a ".test" file in each of the storage groups, *except* it does not in the Music and MusicArt groups. So, is there some step that I am missing to activate a storage group beyond simply creating them? I expect that the problem is some simple step that I haven't done, and like most problems, finding the missing step is 95% of the problem.

Thanks in advance!

marcus hall
Last edited by marcus_hall on Wed Aug 03, 2016 9:43 pm, edited 1 time in total.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Music storage group

Post by paulh »

marcus_hall wrote:I am running the distributed packages from fedora, and recently upgraded to fedora 24, which brings mythtv 0.28. Everything seemed to go well with the upgrade, with one problem. After the upgrade, the music plugin was not working, complaining that the Music storage group needed to be defined. OK, shouldn't be a big deal. I went into myth-setup on the backend, went to storage groups, and defined the Music storage group to be "/var/lib/mythtv/music/", which is the directory holding the music tree. Tried again (I think I was importing files), and it complained about MusicArt not being defined. OK, define that and created a directory "/var/lib/mythtv/musicart". Empty for now, and I cannot find anything on what the directory layout within it should be.. Currently, I have "albumart.jpg" files in each album in the music tree, but it appears that this will no longer be the standard.. OK, I'll figure that out eventually...
marcus hall
The MusicArt storage group is mainly used to store a copy of embedded album art which is extracted from the files tag if it's available. It will also save any artist images and genre icons that you download in the frontend. You only need to make sure the MusicArt storage group has a directory defined and the backend can read/write to it. Any other directories will be created as needed.

You can still use your "albumarg.jpg" in the same directory as the tracks. The file scan will pick them up as before and use them.
Anyhow, now importing files no longer complains about missing storage groups, but still after claiming to have added new songs to the database, I get messages that the copy into the music tree has failed. Indeed, the files did not make it into the tree, and actually the database was not updated. Running the backend with -v file:debug, I see an attempt to delete the file first, which fails saying it doesn't exist. Seems normal, but then when attempting to start the copy:

2016-08-01 12:17:38.737943 D [8924/9041] ProcessRequest storagegroup.cpp:687 (FindNextDirMostFree) - SG(Music): FindNextDirMostFree: Starting
2016-08-01 12:17:38.737956 E [8924/9041] ProcessRequest storagegroup.cpp:726 (FindNextDirMostFree) - SG(Music): FindNextDirMostFree: Unable to find any directories to use.
2016-08-01 12:17:38.737962 E [8924/9041] ProcessRequest mainserver.cpp:1867 (HandleAnnounce) - MainServer: Unable to determine directory to write to in FileTransfer write command


So, I believe that the backend somehow isn't using the storage group definition, or something. When I
attempt to play a song, I see the backend doing:

2016-08-01 20:44:56.128467 I [13325/13375] ProcessRequest mainserver.cpp:3795 (HandleQueryFindFile) - MainServer: Looking for file 'Stewart, Al/Between The Wars/03-Sampan.flac' on host 'brinton' in group 'Music' (useregex: 0, allowfallback: 1
2016-08-01 20:44:56.128477 I [13325/13375] ProcessRequest mainserver.cpp:3800 (HandleQueryFindFile) - MainServer: Checking local host 'brinton' for file
2016-08-01 20:44:56.128964 D [13325/13375] ProcessRequest storagegroup.cpp:608 (FindFile) - SG(Music): FindFile: Searching for 'Stewart, Al/Between The Wars/03-Sampan.flac'
2016-08-01 20:44:56.128991 E [13325/13375] ProcessRequest storagegroup.cpp:622 (FindFile) - SG(Music): FindFile: Unable to find 'Stewart, Al/Between The Wars/03-Sampan.flac'!
2016-08-01 20:44:56.129339 I [13325/13375] ProcessRequest mainserver.cpp:3946 (HandleQueryFindFile) - MainServer: File was not found

The file in question definately exists under /var/lib/mythtv/music, so again it appears to be something related to storage group definition. I can play videos just fine, with the storage group similarly named /var/lib/mythtv/videos.

Now, both of these directories are NFS mounted as well, so if there's any way to just turn off storage groups and go with NFS, I'm good with that, but mythtv seems to be all into reinventing remote file sharing one more time, but, whatever..

I did notice that during startup, the backend seems to create a ".test" file in each of the storage groups, *except* it does not in the Music and MusicArt groups. So, is there some step that I am missing to activate a storage group beyond simply creating them? I expect that the problem is some simple step that I haven't done, and like most problems, finding the missing step is 95% of the problem.

Thanks in advance!

marcus hall
You need to use "Scan for new Music" not "Import Music" they are two different things.

If you manually copy your music files into the "Music" storage group or simply want to rescan the existing music directory then use "Scan for new Music".

If you have some new track and want to add them to your "Music" storage group then use the "Import Music" option. That will scan the directory you tell it for new music and move it into your "Music" storage group even if it's on a remote backend. It will create the directory structure based on the settings you have for the filename format etc. It's mostly useful if you buy some new music online which you download to a directory on the FE and want to move it to the BE music storage group.

If you have problems there is a 'Reset Database" option in the Music settings that will clean up the database allowing for a fresh start. You will loose your playlists though and possibly any manual edits, last played and played counts unless you have told MythMusic to always update the tag with the new metadata.
marcus_hall
Newcomer
Posts: 5
Joined: Tue Aug 02, 2016 1:09 pm
United States of America

Re: Music storage group

Post by marcus_hall »

paulh wrote:
You need to use "Scan for new Music" not "Import Music" they are two different things.

If you manually copy your music files into the "Music" storage group or simply want to rescan the existing music directory then use "Scan for new Music".

If you have some new track and want to add them to your "Music" storage group then use the "Import Music" option. That will scan the directory you tell it for new music and move it into your "Music" storage group even if it's on a remote backend. It will create the directory structure based on the settings you have for the filename format etc. It's mostly useful if you buy some new music online which you download to a directory on the FE and want to move it to the BE music storage group.

If you have problems there is a 'Reset Database" option in the Music settings that will clean up the database allowing for a fresh start. You will loose your playlists though and possibly any manual edits, last played and played counts unless you have told MythMusic to always update the tag with the new metadata.
Yes, this system is upgraded from 0.27, which was working properly. The various database tables are populated with the existing files, and I have a playlist up on the screen, but the frontend is unable to access the music file to play it. Are you saying that I need to do the rescan to complete the upgrade? I am concerned that if I do that, the frontend will not see any files and will then delete everything in the database. Before I do that, I'll have to get a script to export and re-import my playlists, since resetting the database will assign totally new song IDs to everything (as you noted for resetting everything).

The attempt to import files is a separate operation from trying to play music, I was just trying other things out as well, and importing music also seems to be unable to see the storage group since the copy into the music tree was failing.

So, I still believe that for some reason I cannot see the Music storage group. Is there some way to FUSE mount the storage group, or something to allow access to it apart from the frontend to poke around and verify that it is working? Right now, it's sort of a black box that is hard to see into.
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Music storage group

Post by paulh »

Lets get some extra debugging from the backend to see where it is looking for the music files. I think adding 'file' to the verbose flags with be sufficient. You can change the logging verbose flags without having to restart the backend by running

Code: Select all

mythbackend --setverbose file
Don't forget to restore it back after you have done debugging with something like

Code: Select all

mythbackend --setverbose general
Play a music track in the frontend and look at the backend log for any clues.
marcus_hall
Newcomer
Posts: 5
Joined: Tue Aug 02, 2016 1:09 pm
United States of America

Re: Music storage group

Post by marcus_hall »

Presumably that is the same as running the backend with -v file:debug option? With that option, trying to play a song results in:

2016-08-01 20:44:56.128467 I [13325/13375] ProcessRequest mainserver.cpp:3795 (HandleQueryFindFile) - MainServer: Looking for file 'Stewart, Al/Between The Wars/03-Sampan.flac' on host 'brinton' in group 'Music' (useregex: 0, allowfallback: 1
2016-08-01 20:44:56.128477 I [13325/13375] ProcessRequest mainserver.cpp:3800 (HandleQueryFindFile) - MainServer: Checking local host 'brinton' for file
2016-08-01 20:44:56.128964 D [13325/13375] ProcessRequest storagegroup.cpp:608 (FindFile) - SG(Music): FindFile: Searching for 'Stewart, Al/Between The Wars/03-Sampan.flac'
2016-08-01 20:44:56.128991 E [13325/13375] ProcessRequest storagegroup.cpp:622 (FindFile) - SG(Music): FindFile: Unable to find 'Stewart, Al/Between The Wars/03-Sampan.flac'!
2016-08-01 20:44:56.129339 I [13325/13375] ProcessRequest mainserver.cpp:3946 (HandleQueryFindFile) - MainServer: File was not found

as noted in the first message.. (The frontend attempts to play various songs in the playlist, so the above sequence is repeated for each song until it gives up for too many failures.)

This isn't particularly helpful. I do know that the file *does* exist within the music directory, but it seems that there is a problem in the storage group lookup...
User avatar
paulh
Developer
Posts: 909
Joined: Thu Feb 06, 2014 6:09 pm
Great Britain

Re: Music storage group

Post by paulh »

OK I would expect it to show it searched at least one directory something like this
(FindFileDir) SG(Music): FindFileDir: Checking '/var/lib/mythtv/music/' for '/var/lib/mythtv/music/Stewart, Al/Between The Wars/03-Sampan.flac'

So that would seem to suggest there are no usable directories set up for the 'Music' storage group on 'brinton'. Was there any error messages earlier in the log relating to the 'Music' storage group or that directory?

I would double check the storage group in mythtv-setup is present and correct.

Alternativly if you know MySQL another way to check would be

Code: Select all

select * from storagegroup where groupname = 'Music';
marcus_hall
Newcomer
Posts: 5
Joined: Tue Aug 02, 2016 1:09 pm
United States of America

Re: Music storage group

Post by marcus_hall »

Aha! Brilliant!

The problem was my methodology.. I have three "hosts" running on the system, the backend 'brinton', a front-end for the tv 'brinton-tv' and a front-end for access via vnc 'brinton-vnc'. This is intended to allow configuration for the two front-ends to be managed separately. It seems that when I ran mythtv-setup, I had forgotten about this and ran it through a vnc connection, so it created the storage group rows with host 'brinton-vnc', which then isn't seen by the backend with host 'brinton'. So, I've got to always remember to configure the backend from within the backend's world and I suspect all will be fine.

Have updated the tables and will re-try things tonight. I expect it will go a lot better! Thanks!
marcus_hall
Newcomer
Posts: 5
Joined: Tue Aug 02, 2016 1:09 pm
United States of America

Re: Music storage group

Post by marcus_hall »

Yes, that did cure the problem.. Many thanks!
timblayd
Newcomer
Posts: 2
Joined: Fri Aug 03, 2018 8:21 am
Bangladesh

Re: SOLVED: Music storage group

Post by timblayd »

I would suggest to try MusConv tool.
Post Reply