Transcoding jobs not working all of a sudden

[bklein]

Thanks for the input.

journalctl -xe gives the following. I've tried the job a few times and these messages seem to coincide with that. I've tried googling the messages but haven't found any useful posts. EDIT: Ignore this code. I realised is has nothing to do with the user job and is related to two of my tuners not being connected at the moment.

Code: Select all

Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 19
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 19
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 20
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 20
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 21
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 21
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 22
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 22
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 23
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 23
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 24
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 24
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 25
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 25
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_STATE) Unknown encoder: 26
Dec 13 12:43:14 HTPC mythbackend[1213]: mythbackend[1213]: E ProcessRequest mainserver.cpp:4929 (HandleRemoteEncoder) MainServer: HandleRemoteEncoder(cmd GET_SLEEPSTATUS) Unknown encoder: 26

systemctl status mythbackend-service gives the following. Is there some installation / configuration I need to do to get useful output?

Code: Select all

Unit mythbackend-service.service could not be found.

[bill6502]

The default service name in Ubuntu is mythtv-backend. Just using that will
automatically append .service or you can use the full name:
mythtv-backend.service

[bklein]

OK. Thanks. I had a look at the other backend job options as suggested and discovered that this job was not marked to execute on this backend (my only backend), which explains the perpetual queued status. I enabled it and reran the job, but it is failing and I can't figure out why. The logs are not much help. They (journalctl and mythbackend.log) show the following -

Code: Select all

Dec 13 15:08:31 HTPC mythbackend[17368]: mythbackend[17368]: I UserJob_77145 jobqueue.cpp:2440 (DoUserJobThread) JobQueue: Started Transcode H264 for "The Garden Gurus" recorded from channel 80090 at 2020-12-05T05:43:00Z
Dec 13 15:08:31 HTPC mythbackend[17368]: mythbackend[17368]: E UserJob_77145 jobqueue.cpp:2475 (DoUserJobThread) JobQueue: User Job '/home/ben/Scripts/myth_h264_transcode  --starttime 20201205164300  --chanid 80090  --title 'The Garden Gurus'  --outdir /home/ben/Videos' failed.

I have tried no quotes, single quotes and double quotes around the %TITLE% but that makes no difference. It would be useful to know which type of quotes are required around the title variable. Note that the single quotes around the entire command are inserted by MythTV, the are not part of my job command.

Is there any way to get more detail on the job failure or to figure out why it is failing ?

[bill6502]

Double quotes.

I assume this works from the command line (run as user mythtv):

/home/ben/Scripts/myth_h264_transcode --starttime 20201205164300 --chanid 80090 --title "The Garden Gurus" --outdir /home/ben/Videos

Be sure that /home/bin/Videos is writable by user mythtv.

[bklein]

Thanks.

OK, when running

Code: Select all

/home/ben/Scripts/myth_h264_transcode --starttime 20201205164300 --chanid 80090 --title "The Garden Gurus" --outdir /home/ben/Videos

from the command line the job terminates straightaway and the log file shows the following -

Code: Select all

2020-12-13 18:17:39.742643 E  Couldn't find recording for chanid 80090 @ 2020-12-05T16:43:00Z
2020-12-13 18:17:39.742661 I  PowerDBus: Closing interfaces
2020-12-13 18:17:39.743084 I  Exiting

The name of the file is -

Code: Select all

lrwxrwxrwx 1 ben ben   62 Dec 13 18:00 The Garden Gurus - 2020-12-05, 4-43 PM - Untitled.ts -> /media/2TB-01/MythTV-PrimaryRecordings/80090_20201205054300.ts

Note the 054300 in the filename. MythTV has somehow named the file with a start time that is 11 hours earlier than the 164300 actual start time.

I made a copy of the file, naming it with the 164300 start time -

Code: Select all

/media/2TB-01/MythTV-PrimaryRecordings/80090_202012164300.ts

but got the message -

Code: Select all

2020-12-13 18:33:36.826471 E  Couldn't find recording for chanid 80090 @ 2020-12-05T16:43:00Z

Any further ideas on how to troubleshoot this ?
The file is in the MythTV storage location and MythTV would have read rights -

Code: Select all

ben@HTPC:~$ ls -lah /media/2TB-01/MythTV-PrimaryRecordings/ | grep 80090_202012164300
-rw-r--r-- 1 ben             ben              198M Dec 13 18:33 80090_202012164300.ts

And likewise for the output location -

Code: Select all

drwxrwxrwx  2 ben   ben   4.0K Dec 10 03:27 Videos

[daraden]

The crf and preset defaults should be fine, I use similar for HD recordings. The file size increase is at least partially due to de-interlacing.

For the user job try using %STARTTIMEUTC% insted of %STARTTIME% in your command.

[bklein]

Thanks for the info and help.

Using STARTTIMEUTC gave the right time for the filename, but the job still failed from MythTV.

From the mythbackend.log -

Code: Select all

 User Job '/home/ben/Scripts/myth_h264_transcode  --starttime 20201205054300  --chanid 80090  --title  "The Garden Gurus"  --outdir /home/ben/Videos' failed.

Setting the logging level to debug did not give any more information.

However, running the above code from the command line worked.

What could be going wrong that it works from the command line but not from MythTV? Permissions would be an obvious cause, but those seem ok based on what I pasted above (?).

[halucigenia]

bklein, It does not make sense to make a copy of the file with a different name as you did, as Mythtv will not recognise it. Instead I think you should use the time from the actual file name to test.
i.e.

Code: Select all

/home/ben/Scripts/myth_h264_transcode --starttime 20201205054300 --chanid 80090 --title "The Garden Gurus" --outdir /home/ben/Videos

That reminds me that the parameter for the start time that I use in my mythtrascode jobs when I set them up in Mythtv is

Code: Select all

--starttime "%STARTTIMEUTC%"

E.g. the user job in Mythtv that I set up to thest this ruby script is (but see the modifications to the script below before running it)

Code: Select all

/usr/local/bin/myth_h264_transcode_ruby --chanid "%CHANID%" --starttime "%STARTTIMEUTC%" --title "%TITLE%" --subtitle "%SUBTITLE%"  --outdir "/srv/remote/hge01/library/temp/videos/"

Note that I have changed the name of the ruby job and copied it to /usr/local/bin/ and my output directory is a temporary storage space that I use on a machine other than the one that I am running the job from. It may be faster to transcode to another drive than the recording drive on the same machine that your recordings are on, however since I am transferring my encodes to this other machine and filing them in various directories on that machine anyway I don't feel it hinders the process in the long run. Change these parameters to suit your setup.

Now, I have also tested the altered ruby script as a user job in Mythtv and discovered that it does not work, I think that's because of that error message

Code: Select all

51: warning: assigned but unused variable - val

I managed to suppress that by changing

Code: Select all

#!/usr/bin/ruby -W

to

Code: Select all

#!/usr/bin/ruby -W:no-deprecated

The better option would to amend the code to account for this error, but I am not sure how to do that myself.
My guess would be that it is picking up that not all of the potential parameters that can be sent to the script are used?

To get your transcoded file to an acceptable size you will have to alter the ffmpeg parameters in the ruby script.
My preferred options are

Code: Select all

quality     = '20.0'

Code: Select all

#{vFilter} -vcodec libx264 -preset slow -tune film -profile high 

But these can be adjusted to suit what you feel is acceptable.
For reference; This reduced a 17min HD recording from ~960Mib to ~371Mib, thoug the recorded program had no advert breaks and I only cut the start and the end of the recording by ~2 minutes each.

[bklein]

Yes, that warning would be another likely cause of the job not running from MythTV.

I tried changing to

Code: Select all

#!/usr/bin/ruby -W:no-deprecated

but that gave me the following error (I tried with lowercase and uppercase W) -

Code: Select all

Traceback (most recent call last):
/usr/bin/ruby: invalid option -:  (-h will show valid options) (RuntimeError)

Not sure why I'm getting that error and you're not.

[halucigenia]

bklein,
What Version of ruby are you using, maybe it depends on that?

Code: Select all

ruby -v

I am on ruby 2.7.1p83

How about just removing -W altogether?
just #!/usr/bin/ruby
Does that work?

If not, just get it running on the command line before trying to set up a Mythtv job.

[bklein]

Hmmm

I'm on ruby

Code: Select all

ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]

but am up to date on all my packages (Ubuntu 18.04).

Using just #!/usr/bin/ruby worked on the command line but still failed as a user job

[daraden]

try using

Code: Select all

sudo su mythtv -c  '/home/ben/Scripts/myth_h264_transcode  --starttime 20201205054300  --chanid 80090  --title  "The Garden Gurus"  --outdir /home/ben/Videos' 

This will run the command as the mythtv user.

[blm-ubunet]

userjobs run as backend user & can be set to run on the recording BE or the MBE.

The script likely fails because the BE user does not have write access to /home/ben/etc..
Bill6502 also mentioned this some posts back..

This script should make a backup & replace the recording with cut file, then (backup &) delete the cut-list & rebuild seektable & then send FE a popup message when done. The script I posted a link to previously does these things.

To suppress the unused var error change: val to _val
_val = args[1]
eval("#{option} = _val")

[bklein]

@daraden - Thanks. The user job is working now. Running it from the command line as the mythtv user revealed that mythtv did not have access to the folder I specified for the log file (as blm-ubunet has pointed out). I fixed that and the user job is running now.

@blm-ubunet - Thanks. I will try those changes to suppress the val error. It would be great if this script could replace the recording with the cut file. When you say you posted a link to a script that does these things, do you mean this? https://gist.github.com/BipedalTV/3174241