Compression plugins unable to compress before ilab uploads

To reword, this is what I’m hoping for

1. Image is uploaded
2. The optimization plugins are triggered and optimize the image and its thumbnails
3. Media Cloud then syncs the image to the spaces bucket

In order for this to happen, what hooks and actions do these optimization plugins need to utilize to get in front of the Media Cloud sync’ing? And how can they properly reference the sync’d image file path if there needs to be a reoptimization trigger?

These plugins also sometimes generate webp images and then they rewrite the img tags to utilize them, what do they need to do so that the webp images are uploaded as well and that the img tags are rewritten to source from the new URLs from spaces? What do they table do they need to reference to get the correct URL?

Media Cloud inserts itself in the upload chain at a level of 10000. These plugins will have to insert themselves < 10000.

However, if they aren’t overwriting the original uploaded file then Media Cloud probably won’t see the optimized versions.

I’ll be honest, it’s not likely I’ll change much here to accommodate these plugins. I use Imgix, I tell others to use Imgix, all of my clients use Imgix. Imgix takes care of all this stuff for you plus future proofs all of your assets without having to jump through hoops to get it in line with any front end changes you’ve made. It’s $10 a month and you can slap Cloudfront in front of it to avoid CDN charges.

That said, it is open source and I’m open to pull requests ??

Okay that is helpful! I am trying to see how I can get the plugin others to insert themselves <1000, which would overwrite the original uploaded file.

If a plugin is creating WebP format images, is that process outside of the normal image operations that Media Cloud would detect? In other words, how do I make sure that WebP images are uploaded as well? And once all these files are uploaded, how can these plugins access these images that are uploaded? Are the file paths stored in a standard WordPress table?

The reason I’m doing is this way is so that I can also save storage space, this client I have is a publication that is very image heavy – almost 20 images an article and expanding to 8 articles a week.

I haven’t really considered Imgix yet so I think I don’t quite understand it’s pricing model. It says $3/1000 + 8cents GB. I sounds like a CDN like Cloudfront would reduce the bandwidth usage but where are you getting $10/month? They store files as well?

To clarify this is what I’m asking. Please take a look

https://cl.ly/a4e17d980f53

WordPress writes the url of the image into the database and stores the image into uploads
You can ignore the ShortPixel block, this is only if, as you said, the priority is <1000

Now my question is, Media Cloud pulls the url/location from Database, then writes the new location to the same table (supposedly) and grabs the image from /uploads and uploads it to S3.

What I’m confused about, that if Media Cloud is writing the location back to the same table, why is it that a plugin like ShortPixel, resmushit, etc is unable to optimize the image? They both error out as if the image is not there, but they should be able to just optimize the image from the object storage shouldn’t it?

I also did a database search and couldn’t find URL of the digital ocean spaces endpoint anywhere – what table is it stored?

I think I figured out what’s going on, maybe you can confirm.

I looked at the _wp_attached_file meta_key within wp_post_meta and it looks like it only shows the relative path so it’s never changed – that’s good to know.

And I figured out where Media Cloud permanently changes the domain to the bucket endpoint, upload_url_path in wp_options, do you have documentation about this somewhere?

Now I suppose to deal with the four issues I have now

1. Shortpixel/image optimizer needs to optimize the images in a higher priority than 1000
2. Shortpixel needs to be able to know that the image is stored somewhere else – how can pixel find the new physical location information
3. Shortpixel needs to be able to read/write, is there an api in which another plugin could access Media Cloud’s s3 storage auth system?
4. How is it some images are utilizing the original server location while some are not? After I changed upload_url_path, the file_url was changed for the majority of images so how is it deciding which is the bucket endpoint and which is the original server?
5. Is there an option to revert the file_url within Media Cloud’s settings?

• This reply was modified 6 years, 4 months ago by Tanner.
• This reply was modified 6 years, 4 months ago by Tanner.
• This reply was modified 6 years, 4 months ago by Tanner.

Media Cloud doesn’t change anything in the database other than adding an ‘s3’ entry to an attachment’s meta. It calculates the URL every time based on the info in the ‘s3’ meta field.

Turning off the plugin will restore URLs. Whether or not those URLs will work depends on if the media is still on your WordPress server. However, if you are using Imgix and have uploaded stuff since you’ve started using Imgix, you can’t go back without having to replace that uploaded media because when Imgix is enabled the WordPress thumbnails are not uploaded to S3.

The plugin definitely doesn’t change any wp_options. The upload path is calculated every time a file is uploaded. Prior to 2.1.6 the upload path was calculated any time upload_info() was called, but in 2.1.6 it’s been changed so that the path is only calculated during the upload process. You can have entirely dynamic upload paths with Media Cloud so there’s no choice for it to be anything but dynamic.

I don’t know what effect setting upload_path or upload_url_path in wp_options will have. If you grep update_option in the code base you’ll see it’s only used to set ilab* options. I honestly don’t even know what upload_path and upload_url_path do. The sites I took a look at today all have empty values for these options and autoload set to true.

You can get access to the storage interface via:

$manager = \ILAB\MediaCloud\Tools\ToolsManager::instance();
if ($manager->toolEnabled('storage')) {
    /** @var \ILAB\MediaCloud\Cloud\Storage\StorageInterface */
    $client = \ILAB\MediaCloud\Cloud\Storage\StorageManager::storageInstance();
    .. do your thing ..
}

And if these plugins hook into wp_handle_upload filter and alter the $upload argument, Media Cloud will use that value. For example, if the plugin did something like this:

add_filter('wp_handle_upload', function ($upload, $context = 'upload') {
                $upload['file'] = '/some/other/thing.jpg'; 
                return $upload;
            }, 9000);

Then Media Cloud will use '/some/other/thing.jpg' for processing.

Or just pay the $10 (Imgix’s monthly minimum charge), put Cloudfront in front of Imgix and call it a day ??

The $3/1000 is being charged $3 per 1000 *master* images that Imgix pulls from your S3 within a month. If you pull 4000 images in a month, it’s $12.

This plugin was originally built for a site that has about 70,000 images on it (https://milk.xyz/) and our monthly bill is rarely above $15.

I mean imagine having to regenerate thumbnails for 70,000 images.

Ah that makes a lot of sense and thanks for detailing all of that. What it sounds like is that these optimization plugins would need to utilize that s3 table entry and then have the ability to access the s3 bucket on their own.

But is it possible for Media Cloud to dynamically intercept any plugins that might need to access the media on s3? Since Media Cloud is the only plugin that is aware of these new s3 paths and that can access the s3 bucket, it would be great if it could just take over the media read/write in general. This way something like ShortPixel just requests the image and Media Cloud intercepts and returns the image based off of the dynamic path and the ability to access s3.

Or for example, if I use a plugin to generate thumbnails before turning off Imgix, then the thumbnail generator would retrieve from the bucket and write new thumbnails to the bucket with upload_info by Media Cloud and be completely agnostic to the storage or Imgix. Then I’d be able to turn off Imgix safely.

As another precaution, would you be willing to add a features that would allow redownloading all of the images if they were chosen to be deleted at any stage?

This site has 93k so that’s good to know about the estimate going to be about $15.. I’ll have to test it out.

I actually tested a lot of these plugins today and found out that all of them try to do the image processing in the background AFTER the upload is a done deal. Obviously this doesn’t work with Media Cloud and I don’t see how it could unless they insert themselves into the upload flow. But it might be they can’t do that because the image optimizations take so long.

So the non-optimal solution these plugins, or plugins like them, will have to use is downloading from S3, optimizing and uploading back to S3. Which means every upload will be a 2 uploads to S3 + 1 download from S3.

In 2.1.8 I add error and warning notices to the WordPress admin that you are using incompatible plugins with a description as to why. All of the image optimization plugins I tested were added to that.

Yeah you’re right, they don’t set their priorities high enough to be in front of media cloud but I’ve been chatting with ShortPixel to see if it’s possible they could change that. I personally don’t mind the optimization wait time once they do so it would still only be one upload.

Once I get them to then media cloud wouldn’t need to do anything in terms of upload but would you consider having media cloud intercept any read and write executions from other plugins in case I want to have ShortPixel generate thumbnails as a backup to turning off imgix (for debugging or other reasons)?

Image optimizer support is now live. ShortPixel, Imagify, EWWW and Smush.

Imagify is the best of the bunch, imho. ShortPixel has some issues that aren’t specific to Media Cloud, for example using WordPress’s image editor doesn’t trigger ShortPixel to do anything while Imagify automatically optimizes the new image edits. Also, ShortPixel requires your service be publicly accessible, while Imagify doesn’t. That kind of sucks for a dev setup, but also because of how I usually setup clients WordPress installs:

Comment
byu/RO9a0TON from discussion
inWordPress

Anyways, let me know if you run into any issues!

Cheers,

Jon.

PS. Make sure to keep the storage setting “Delete From Storage” turned off, and toggle on the new Imgix setting “Keep WordPress Thumbnails”.

Hi @interfacelab
First of all, congratulation for your great plugin. It’s really awsome.

I’m so happy to see that your plugin now works with “EWWW Image Optimizer”.
I stoped to use “EWWW Image Optimizer” beacuse it was not working with your Media Cloud.

After your update, I installed “EWWW Image Optimizer” again.
Now, The image is optimazed, BUT it’s not send to the cloud. The url of the image keeps in local. Is it a bug or do I have to do someting to optimize image and send it to the cloud automatically when uploading a new image?

@henryavila

Can you tell me what PHP version you are using?