Encoding issues, no log file, no encoding happens

BTW, if I pull the params from your plugin, as mentioned, and test with a new simple script like the below, it WORKS.

<?php

exec(‘/usr/bin/ffmpeg -y -i <site/dir/file>.wmv” -acodec libvo_aacenc -b:a 128k -s 640×480 -vcodec libx264 -threads 1 -b:v 1920k -bt 800k -f ipod “/home/<site/dir/file>-480.mp4″‘);

?>

There are two differences between the way thumbnails are generated and the way encoding works in the plugin and your script doesn’t use either one of them, so if you don’t mind a little troubleshooting I would love your help in getting to the bottom of it.

1. nohup In order to avoid hanging up php while the encoding continues for who-knows-how-long, ffmpeg is run using nohup. It’s possible this is in fact hanging you up.

2. Redirecting ffmpeg’s output to a log file. In order to figure out what ffmpeg is up to at any given time the plugin redirects all output to a log file, which is not being generated. Sometimes the encoding is still happening (have you checked the directory to verify that there are no new files?) but the plugin thinks nothing’s going on because the log file isn’t being generated.

Can you try this from the command line and let me know how it goes?

nohup /usr/bin/ffmpeg -y -i "<site/dir/file>.wmv" -acodec libvo_aacenc -b:a 128k -s 640x480 -vcodec libx264 -threads 1 -b:v 1920k -bt 800k -f ipod "/home/<site/dir/file>-480.mp4 > /home/<site/dir/file>_480_encode.txt 2>&1 & echo $!"

I don’t mind at all.

entering the command (with the three chars $!” at the end) in CLI produces this:

-bash: !”: Event not found

The exact command was:

nohup /usr/bin/ffmpeg -y -i “/…/hmp.wmv” -acodec libvo_aacenc -b:a 128k -s 640×480 -vcodec libx264 -threads 1 -b:v 1920k -bt 800k -f ipod “/…/hmp_480.mp4 > /…/hmp_480_encode.txt 2>&1 & echo $!”

I have an idea there may be an error in this command though, so I will take a look at it, but I think nohup Works. Nice too.

Will write back shortly.

OK, editing the command so the last bit is $! (no quote) and moving the quote to the end of the output filename, results in the system printing the process id and terminating from CLI right away.

I can see that the process continues, so nohup works.

Running it in a browser as the php script below (with and without the end $!)resulted in nothing:

<?php
exec(escapeshellcmd('nohup /usr/bin/ffmpeg -y -i "/.../hmp.wmv" -acodec libvo_aacenc -b:a 128k -s 640x480 -vcodec libx264 -threads 1 -b:v 1920k -bt 800k -f ipod "/.../hmp_480.mp4" > /.../hmp_480_encode.txt 2>&1 $!'));
?>

Changing the php code to not include the escapeshellcmd part AS WELL AS the nohup part results in success – the below works:

<?php
exec('/usr/bin/ffmpeg -y -i "/.../hmp.wmv" -acodec libvo_aacenc -b:a 128k -s 640x480 -vcodec libx264 -threads 1 -b:v 1920k -bt 800k -f ipod "/.../hmp_480.mp4" > /.../hmp_480_encode.txt 2>&1');
?>

Neither attempts with escapeshellcmd in the php line, nor nuhup in the shell part, worked.

So it seems that I must ditch both nohup AND escapeshellcmd, unless I have it disabled in suhosin (don’t think so)

BTW the plugin never generated a log file. It didn’t even write a 0-byte file.

Oh, new findings.
It may be some sort of security issue. I don’t understand why escapeshellcmd makes it not Work (and don’t know if it is so in the plugin too).

But, the nohup solution above works if I remove the “& echo $!” bit.

Interesting. It’s possible on your system you can’t do multiple commands using &. Does it work with && ? I might be able to simplify that command too.

I will check.

But it DOES work in CLI.

I’ve got it working with nohup nice. NOT using the “& echo $!” bit Works.

But now, when a video is encoded, nothing happens in the queue. This also Means the mp4 is not registered by the plugin.

That’s because when you take out the $! part it’s breaking the plugin because it doesn’t get the expected return value (the PID) so none of the remaining functions in that chain get executed. I suspect there’s a better way to deal with this that will be more universal.

Oh, I missed one part of your earlier testing. Running escapeshellcmd is designed to prevent exactly that kind of combining of commands using &. The plugin runs escapeshellcmd before adding the & echo $! bit.

Yeah, I guessed it had to make use of the return value.

Actually:

https://stackoverflow.com/questions/222414/asynchronous-shell-exec-in-php

I think it should Work with semicolon (not &&). It doesn’t though, if there should be a Progress bar (I see none)…. I will check and test and report back.

-Peter

Damn. I modified the test php script to return the PID to the browser.

It did, and it was consistent with the rest of the PIDs running at that time.

So… the exec Works (not a PHP error).
The parent PID is provided by the process (ffmpeg) and returned to PHP, which outputs it in the browser.

At this time, the first process (ffmpeg) has ended.

So the & part Works, as does nohup, nice and the PID printout.

Other people have mentioned the multi process exec() functionality is often not working.
I guess I will try to re-boot my server, and see if tomorrow brings other ideas.

(BTW it could be possible to alter the script so the process id is written in the beginning of the log, or as <orig-filename>-<pid> so the plugin can keep track of it another way.)

— It worked after a reboot, no idea why that was needed, but I had disabled and re-enabled suexec as a test, as well as changing from fcgi to mod_php and back Again.

Right now, it correctly spawns and returns the PID, and also shows a Progress bar.

I’ll keep you informed.

For now, thanks for your help.

– Final edit here: the problem has been identified.
The issue arose again, and my digging in the system logs revealed that /dev/null was reset with permissions that didn’t allow www-data to write to it, breaking the plugin. It seems some function in the plugin still exports the log to /dev/null which I suggest you redirect to a dummy log file?

-Peter

Interesting. I can’t find anything that explicitly writes to /dev/null but I’ll keep digging.

I think it is other running CRON tasks that somehow hook into the log file for your script.
In the end (just before an error occured), there was a cron output from my wp-cron, which writes to /dev/null. I have changed that so it writes to a file, to see if the problem persists.

This caused your plugin to THINK the encode had halted due to an error. This was not the case; the process list revealed that they continued to encode.

Your log checker routine btw requires an additional regex check to make watermarking (another support thread) work.

I did (around line 2960):
preg_match(‘/Parsed(.*?) /’, $lastline, $buffer_matches);
to workaround this.

I wouldn’t mind sending you my edits so you can test them for a later version (no idea in inventing the deep dish twice) ??