mythfilldatabase not updating?

[tknorris]

In hopes of preventing someone from spending the week of confusion I just spent trying to figure out why mythfilldatabase wasn't updating my listings automatically via Schedules Direct I thought I'd share the problem I had and the solution.

Symptom: When mythfilldatabase is run manually it works, but when scheduled to run automatically it does not. Similarly, when run from one userid it works correctly, but when run from the mythtv id it appears to run correctly (i.e. no unexpected errors), but never updates any schedule data. in addition, the mythfilldatabase log shows that the schedule xml downloaded from Schedules Direct does not contain any data for the new days being requested when run from the mythtv id. Edit: Also, I forgot to mention that the mythfilldatabase log will report in both runs that the compressed and uncompressed byte counts of the downloaded schedule data xml are identical, but when processing the files it will claim they contain data for different days!

Problem: When mythfilldatabase is run using Schedules Direct the requested schedule xml is stored as /tmp/mythtv_ddp_data with 755 permisions and owned by whatever user mythfilldatabase is running as. As a result, if mythfilldatabase is run from some other ID (in my case, xbmc), any subsequent run via the mythtv ID will not be able to overwrite the /tmp/mythtv_ddp_data schedule xml that was created by the other id. Thus each run will just keep using the downloaded schedule data from the previous run.

Solution: Delete /tmp/mythtv_ddp_data and only run mythfilldatabase from the mythtv userid. (There may be other solutions as well, but this is the simplest.)

Note though, It's still not clear to me why mythfilldatabase doesn't throw an error when it can't overwrite the existing schedule xml or why it doesn't cleanup the downloaded schedule data after each run is complete.

[bill6502]

#11195 was opened to address this. However, I was no longer able to duplicate it and suggested closing it. It might help if you added two more details.

What version of MythTV are you running and how many sources do you have?

I'm thinking that when originally opened, I had a single OTA source and later added a second cable source and that may be why the problem appeared to go away.

[tknorris]

I'm running MythTV 0.27 from the Mythbuntu 0.27 PPA (https://launchpad.net/~mythbuntu/+archive/0.27) and I only have one source.

[bmk1979]

I can confirm this, tknorris's post just made my day. On a new myth 0.27 install with one source, I ran mythfilldatabase logged in as one user, and have been running the backend and mythfilldatabase as a different user ever since, and getting no guide data. When I would run mythfilldatabase manually, it did not complain, but it would output lines like the following:

Grab complete. Actual data from 2014-02-15T23:59:00Z to 2014-02-15T23:59:01Z (UTC)

Notice that the difference in dates/times is only one second.

I found tknorris's post, deleted /tmp/mythtv_ddp_data, and reran mythfilldatabase as the correct user (the one which runs mythbackend and mythfilldatabase) and all is well.

[tknorris]

Glad to help someone else out. It took me so long to figure out what was wrong I couldn't just keep it to myself.

Bill, As far as the fix described in the ticket, I'm not sure that's the best way to go. IMHO, trying to change the permissions isn't going to work 90% of the time because the problem is going to be file ownership not actually permissions. Thus, the attempt to set the permissions is going to fail almost everytime because the file is not owned by the ID that mythfilldatabase is currently running as. Better to try to avoid the problem than fix it after it happens.

It would make more sense to me to:

1. Issue an error anytime an attempt to write the schedule data fails for any reason, thus at least making the problem more obvious if it occurs.
2. Clean up any downloaded schedule data at the end of each run (thus it won't be present in subsequent runs for another id). This way the only way you can get the problem is if mythfilldatabase is ctrl-ced or core dumps.

[bill6502]

I retested on my 'test' host which is now back to a single source and the stale file remains. In 0.28-pre, its name has changed to /tmp/mythtv__ddp_data (extra _.) The developer's solution cared for the problem by (as near as I can tell) keeping the input in RAM rather than a file. And, as mentioned in the ticket, I don't believe it was ever implemented. I'd say there's enough proof that the issue remains in the single source case that I'll reopen the ticket after trying to collect some more data. Note the developers will sometimes prefer a new ticket.

Side note: for the multiple source case, the input file name looks like

Code: Select all

/tmp/mythtv_ddp_Qjggn2/mythtv_dd_cache_1_<yourSDlogin>_UTC_20140214192717_to_20140303192717

and gets removed.