Builds

For discussion of topics specific to MythTV on OSX
sfatula
Senior
Posts: 163
Joined: Sun Aug 09, 2015 3:51 am
Location: Calera, OK
United States of America

Builds

Post by sfatula » Fri Apr 06, 2018 4:50 am

For background sake, I know warpme maintains Macports for mythtv and the associated all in one installer. I know he spends a lot of time on this and should be commended for the great job. I know there are a number of limitations of the macports "builds", at least for me. Craig can't possibly chase them all down. There also exists other builds that people post from the compile script, but those have other issues and are a no go for me.

So, I spent a bunch of time making my own build and fixes for 29.1 front and back (currently missing from other builds) that work with plugins, python, perl, qt, etc. I had thought that perhaps I could contribute it. Problem is, it doesn't use either the macports dirs, or the all in 1 dirs. I just have an alternate way of looking at versions and builds than Macports does, provides more control. Seems like the whole thing would be more reliable, stable, and consistent.

That being said, very little of what I have fixed or done will mean anything to Macports and others builds as it's based on some fundamental differences. I am not sure of the value of packing up an installer and doc. Too many other mythtv for Mac builds, might just confuse support more. My idea had been to provide an installer, install in user and standard dirs where it's easily accessed, and fix issues I could find within that strategy. i.e., a very simple install process that's more the Mac way, next to 0 install steps for backend or frontend. Still, not certain of any value here (except to me, loving it!).

I had also wanted to take some of the OSX patches and submit into the codebase, not sure if Craig wants to do that. Such as the qt changes that avoid focus issues. Maybe that would be of value as patches would no longer be necessary. I can do the code.

Anyway, I was all ready to contribute, but not sure of any value. Seemed like if mythtv could just provide a Mac installer that is simple, things might be better. I guess I am not sure I should go any further, except for my purposes and use. And I don't want to make what may seem like competing builds. People using Macports or all in 1 likely will desire to stay with those. Thoughts?

I might also want to add a few code features that I could at least use, but likely others (not Mac specific). Craig, is there someone in the project or a forum area where it might be best to approach? i.e, wanting to volunteer some enhancements but I am sure developers or architect want to make sure it fits within the goals.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Fri Apr 06, 2018 2:49 pm

If I'm following you, you are addressing 3 interrelated issues:

Mac-specific patches -
I noticed some time ago that Warpme maintains an extensive set of patches to MythTV. He was kind enough to provide an overview of them. In the end, I think I only applied one or two in MacPorts. For me, a key limiting factor is getting support from the MythTV devs. It is hard enough to figure out when Myth on Mac isn't behaving right--it complicates things a lot if there are a raft of patches that the devs have not accepted.

I think the best approach is to advocate with the devs to have key patches accepted into the project and applied to the core code. Jump onto the #mythtv IRC channel and talk to the devs there. Most of these patches are already on tickets in Trac. Point out what you think are the advantages of applying the patch. Try to help address the concerns they may have. Being able to do A-B testing is a big help since, as far as I know, none of the currently active devs has a working Mac development environment.

Build/Install process
As you point out, there are umpteen ways to build, install and/or package MythTV. If you plug away long enough, you can get a working build with pretty much any tool. However, over time, keeping ahead of bitrot is the hard part. A full front-and-backend install needs something over 200 dependencies.

Bitrot was the main issue I had with the original Perl script. Lots of dependencies had become outdated. Specific versions were no longer available in some instances, etc. Also, the script was largely geared to build the latest version of master and sometimes had problems building prior versions.

MacPorts offered a couple of advantages. Other maintainers are on top of keeping up with most of our dependencies. Are there breaking changes? Sometime, but not often and the advantage of having someone else worry about Mariadb and Qt5 and all the rest is worth it, in my opinion.

All this gets multiplied if one wants to support older MythTV versions and older macOS versions. I have already dropped MythTV 0.25 and 0.26 from MacPorts since I have no time or resources to even pretend to support these old versions. The same for ancient versions of OS X.

Install location and format

As you know, the Perl script creates application bundles. That's great for Mythfrontend since it ends up being a quasi-normal Mac application that can be started by double-clicking the app. It makes less sense for MythFilldatabase...and is actively confusing for MythBackend, especially for new Mac users that probably don't have much experience with Terminal. Almost anything they would read on the mailing list or in the wiki would simply not work on their Mac.

MacPorts lets Myth act much more like the Linux versions by putting the software into locations that on the PATH. (And gives the user access to thousands of other open source software packages, as well.)

The ultimate, though, for Mac users is to avoid the command line entirely and just install from a standard dmg or pkg. Again, that was part of the attraction of MacPorts. It includes built-in tools to walk through all the dependencies, de-dupe them, and build an installer. That can certainly be done 'by hand' but is a non-trivial exercise. Especially building preinstall / postinstall scripts that do most of the work on behalf of the user.


I'm not trying to say that MacPorts is the only true way. Any approach has tradeoffs. This is open source software, so we're all free to scratch whatever itch we choose.

Craig
PS I'm not "warpme". I generally use pvr4me or pvr4Craig online.
The MacPorts guy.

sfatula
Senior
Posts: 163
Joined: Sun Aug 09, 2015 3:51 am
Location: Calera, OK
United States of America

Re: Builds

Post by sfatula » Fri Apr 06, 2018 3:34 pm

Oops, wrong name. Perhaps you did not understand or I did not say I've already done all that work. My thought was it might be better for less technical users to be able to install Mac MythTV as any normal app on normal Mac paths, and this already works. All dependencies, etc handled, done. Not like Linux version, easier since no command line access is needed unless you do more advanced things. It works. My question was more of a is there a place for this or not. I will continue to use this forever more as I greatly prefer it and it runs without the issues I encountered with all of the other methods. So, that is a done deal, the question is contribute or not. To me, this belongs in the app store too.

This is not to compete with any other build, just a different choice.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Fri Apr 06, 2018 4:38 pm

I thought you had a working script (based on warpme's) that would download, extract, configure, build and install Myth and it's dependencies. Do you mean you have also automated building a binary installer (pkg or dmg) plus pre / postinstall scripts that instantiate the database and create the mythtv user, etc, etc? That's a tonne of work.

Your scrip uses the default '/usr/local' for the install prefix rather than '/opt/local' that MacPorts defaults to, right? Note that other installers may possibly clobber some of your dependencies. MacPorts uses /opt/local for just that reason.

BTW, what version(s) does your script build? Fixes-29? Master? Fixes-0.28?

If you want to make this available to others, go ahead. I'm not sure how you'd get it into the Mac App store but more power to you if you can. I simply tried to explain the reasons for the choices I made. We already have two very different build methods. One more isn't going to cause the world to fall over.

Craig
The MacPorts guy.

sfatula
Senior
Posts: 163
Joined: Sun Aug 09, 2015 3:51 am
Location: Calera, OK
United States of America

Re: Builds

Post by sfatula » Fri Apr 06, 2018 9:28 pm

It is a ton of work, not perfect yet but getting there. The script uses neither /usr nor /opt, it installs into whatever directory the users wants like other Mac software.

I think I'll talk to the IRC channel as you suggested and see if they want any Mac help. I am one handed for a while so have not done much lately.

I built fixes-29, v29.1 tag. It really works well, no GUI issues. Been using it. Thanks for your thoughts.

I never was able to get all the myth fonts working except by putting them in ~/Library fonts area. Some themes never looked right under Macports, nor mine either but looks right when I put them there.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Sat Apr 07, 2018 12:53 am

You didn't answer, though, whether you're creating a dmg/pkg installer? Are you doing that now? Or is this a direction you want to go (viz the Mac App Store)?

What do you mean "The script uses neither /usr nor /opt, it installs into whatever directory the users wants like other Mac software." I think most users expect that bundled applications are installed under /Applications. Other than that, few users have any idea where other software gets installed. In any event, most open source software obeys an install prefix; that's how virtually all the packages in MacPorts get installed where they do.

Re fonts, my understanding is that Qt uses fonts available via the (Mac) operating system. Thus we need to use something like /Applications/Font Book to register the fonts with the system. Error messages in the frontend log identify missing fonts (over and over and over). However, I've never found anything definitive about the combination of Myth, fonts, Qt and the Mac. If you do, please share.

One thought...you might want to suggest replacing the current perl packaging script with yours. Basically, nobody has touched the perl script in 2+ years. You would need to extend your script so that it could pull the current git master and build from that. (And only recompile/link the changed parts.) The MacPorts build process will never work this way. A key tenet of MacPorts is to build from tarballs (well, preferably to supply pre-compiled software build from exactly that tarball) that can be mirrored throughout our CDN. Building from git head doesn't comply. Having a functional replacement for the perl script would make it easier for a Mac developer to start contributing to the project.

Craig
The MacPorts guy.

sfatula
Senior
Posts: 163
Joined: Sun Aug 09, 2015 3:51 am
Location: Calera, OK
United States of America

Re: Builds

Post by sfatula » Sat Apr 07, 2018 1:05 am

Yes, a package. Stuff goes into Applications and a user directory. The only way I got fonts to work is moving them to ~/Library/Fonts

I am asking them about being that developer if they want/need help.

Gribnif
Junior
Posts: 45
Joined: Wed May 14, 2014 12:46 pm
United States of America

Re: Builds

Post by Gribnif » Thu Jul 05, 2018 2:17 pm

What is the current state of build scripts for version 29 on OS X? I found warpme's pre-compiled frontend, but I also need the backend. It doesn't look like macports includes 29 yet.

I'm willing to compile it myself, but not to go through what I'm sure is dependency hell to get it to work. Has anybody figured it out already?

MoisiePants
Junior
Posts: 25
Joined: Sat Jul 23, 2016 11:41 am
Great Britain

Re: Builds

Post by MoisiePants » Sat Jul 07, 2018 6:59 am

Gribnif wrote:
Thu Jul 05, 2018 2:17 pm
What is the current state of build scripts for version 29 on OS X? I found warpme's pre-compiled frontend, but I also need the backend. It doesn't look like macports includes 29 yet.

I'm willing to compile it myself, but not to go through what I'm sure is dependency hell to get it to work. Has anybody figured it out already?
Hello Gribnif:

I believe I have a working 29.1 build for MacPorts; I just need to do a little more testing (a couple of bugs which I think are core Myth, rather than the build process), and then I'll be passing my updates to Craig for implementing into the MacPorts release.

I hope to have my end done within the next week or two, but I can't comment as to how long it will be before Craig can implement the changes.

Hope that helps,

Pants.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Sat Jul 07, 2018 12:59 pm

MoisiePants wrote:
Sat Jul 07, 2018 6:59 am
I believe I have a working 29.1 build for MacPorts; I just need to do a little more testing (a couple of bugs which I think are core Myth, rather than the build process), and then I'll be passing my updates to Craig for implementing into the MacPorts release.

I hope to have my end done within the next week or two, but I can't comment as to how long it will be before Craig can implement the changes.
What kind of bugs are you seeing?

Craig
The MacPorts guy.

MoisiePants
Junior
Posts: 25
Joined: Sat Jul 23, 2016 11:41 am
Great Britain

Re: Builds

Post by MoisiePants » Sun Jul 08, 2018 9:04 am

Hi Craig:

The main one I'm looking to address is that, on setting up a fresh install, Myth-Setup crashes when it attempts to update the (empty) database schema - there's an options file it's expecting, but which isn't present:

Code: Select all

2018-06-23 08:57:20.824296 E [17380/775] CoreContext dbutil.cpp:722 (DoBackup) - DBUtil: Error backing up database: 'mysqldump --defaults-extra-file='/tmp/mythtv_db_backup_conf_cZBOL6' --host='localhost' --port='3306' --user='mythtv' --add-drop-table --add-locks --allow-keywords --complete-insert --extended-insert --lock-tables --no-create-db --quick 'mythconverg' > '/tmp/mythconverg-1344-20180623075720.sql' 2>/dev/null' (127)
2018-06-23 08:57:23.842842 E [17380/775] CoreContext main.cpp:585 (main) - Couldn't upgrade database to new schema.
I'm still finding my way around the code base, but I'd like to squish this before forwarding my changes. It works fine if you restore a backed-up database - it's just the new database update that fails.

Pants.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Sun Jul 08, 2018 1:45 pm

Hmmm, dbutil.cpp hasn't been touched in 3 years:

https://github.com/MythTV/mythtv/tree/f ... ibmythbase

The Mac version has never been able to run the automatic database backup. I think it expects to find '/usr/local/bin/mysqldump' but, of course, ours is not located there. It has never prevented walking the database schema up to the current version, however.

Craig
I don't think I tested this scenario when I was messing around with v29 last year. I brought a 0.28 database forward.
The MacPorts guy.

MoisiePants
Junior
Posts: 25
Joined: Sat Jul 23, 2016 11:41 am
Great Britain

Re: Builds

Post by MoisiePants » Wed Jul 11, 2018 3:39 pm

Hmmm, dbutil.cpp hasn't been touched in 3 years:
Ah - ok. In which case I'll get my PR finished off without waiting on getting this fixed - at least it won't be in a worse position than it is currently then.

I'd still like to fix this - but building a debugging setup is taking a while for someone who's only used to working on standalone projects in Xcode... ;)
The Mac version has never been able to run the automatic database backup. I think it expects to find '/usr/local/bin/mysqldump' but, of course, ours is not located there.
It looks like it first attempts to use the mythconverg_backup.pl script - but this fails, and then it attempts to run its own internal backup routine - which also fails.

User avatar
bill6502
Developer
Posts: 1086
Joined: Fri Feb 07, 2014 5:28 pm
United States of America

Re: Builds

Post by bill6502 » Wed Jul 11, 2018 4:41 pm

Right, the Perl script 1st (in the shared directory) then mysqldump, which could be a $PATH issue.

User avatar
pvr4me
Senior
Posts: 627
Joined: Fri Feb 07, 2014 7:25 pm
Location: near Toronto, Canada
Contact:
Canada

Re: Builds

Post by pvr4me » Thu Jul 12, 2018 11:17 am

bill6502 wrote:
Wed Jul 11, 2018 4:41 pm
Right, the Perl script 1st (in the shared directory) then mysqldump, which could be a $PATH issue.
Am I right that it uses a hard-coded path to look for the Perl script? Can it be changed to respect PREFIX as defined in configure?

Craig
The MacPorts guy.

Post Reply