what actions cause the plugin to directly access the S3 bucket via access keys

  • We have been using the WP Offload Media Lite plugin on 100+ sites for a decade now. We were evaluating our S3 costs in AWS by looking at data costs involved with the S3 buckets where our images are stored. While doing so – we noticed that the bytes out from a multisite with about 10 sites on it was 50% that of the bytes out from all the rest of our 100 sites. So, in other words – if our 100 sites had 600GB of data out per day, the 1 multisite had 300GB of data out.

    This seemed incredibly disproportional, so we enabled logging on the S3 buckets to attempt to find where the images were being called from. In doing so on the multisite bucket, we found many references where the requester was the AWS ARN for the IAM User associated with the access keys used in the plugin to allow it access to the S3 bucket.

    If the plugin was accessing the bucket directly, I assumed this would be just when the plugin is moving an uploaded media gallery item over to the S3 bucket. I also assumed that any other time the site would reference the media item after this, it would simply use the URL from the as3cf_items table. So any items in the S3 access log using the AWS ARN should be for new media library items.

    However, we noticed that a lot of these references in the log containing the AWS ARN were for old media library items several years old. So my question is – what other actions (aside from the initial upload) would cause the WP Offload Media Lite plugin to access a media item from the S3 bucket directly using access keys instead of just using the public S3 URL for the media item.

    The increase is not due to just the amount of traffic that the sites get. The websites in the multisite do not receive massive amounts of extra visits that could possibly account for the additional resources used. We’re hoping if we can determine what other WordPress actions can cause this – we can find why this multisite in particular is using such a drastic amount of transfer data compared to the rest of the sites in the entire company.

    Thank you so much for any help anyone can provide!!
    Jim

Viewing 1 replies (of 1 total)
  • Plugin Support Delicious Brains Support

    (@dbisupport)

    Hi Jim,

    WP Offload Media Support Team here. Thanks for reaching out with your query, we would be happy to assist!

    We have asked assistance from our dev team about your question. Since you are using the Lite version, there are no tools or other background processes that could be accessing files using the S3 API via our plugin. So we can exclude that.

    Regarding this:

    If the plugin was accessing the bucket directly, I assumed this would be just when the plugin is moving an uploaded media gallery item over to the S3 bucket. I also assumed that any other time the site would reference the media item after this, it would simply use the URL from the as3cf_items table.

    This is correct.

    The one thing they can think of is whether you are using?Remove local media.?If so, when a WordPress user is editing images in the image editor, WP Offload Media downloads the file locally and then re-uploads it again. But we have no idea how this would look like in the S3 access logs. Would it be possible to share a few rows from the access logs so we can look for other more clues?

    In theory, forcing a file download, like what the Image editing process does, could be triggered by other plugins as well. Are you maybe using a plugin that walks through all the media files in the library and does something that triggers a file download from S3? Perhaps you ran an optimization plugin recently?

    It would also help if we could get the diagnostics from the multisite where you are having issues.

    1. Go to the Support tab of your install
    2. Scroll down to the “Diagnostic Info”
    3. Copy the contents of the box
    4. Paste the contents to your reply here

    Feel free to mask out or remove any private information that you don’t want to share. What we need is a list of active plugins in the site that could be causing the issue.

    Looking forward to your response!

Viewing 1 replies (of 1 total)
  • The topic ‘what actions cause the plugin to directly access the S3 bucket via access keys’ is closed to new replies.

Tags